About this role
About the Role
Join an early-stage AI evaluation infrastructure startup as a Research Engineer focused on quality control automation. You will help build systems that make high-quality training data more scalable, combining careful human judgment with reliable measurement and automation.
What You'll Do
-
Build and improve quality control systems for training data created within reinforcement learning environments.
-
Define quality standards and design experiments and metrics to assess agent outputs.
-
Develop auditing approaches, including sampling strategies and rule-based or model-assisted validation pipelines.
-
Work with data vendors to diagnose quality issues, understand agent failure modes, and improve data generation processes.
-
Turn quality control findings into improvements to internal infrastructure and vendor-facing tools.
What We're Looking For
-
Two to four years of experience in a research engineering or similar role, including hands-on work in quality control automation.
-
Proficiency with Python, Docker, and Linux, plus experience building scalable data validation and automated QA/QC systems end to end.
-
Experience with benchmarks and evaluations for reinforcement learning training data, including realistic tasks, reliable rubrics, and useful training trajectories.
-
Ability to define and measure training data quality, and to build quality control systems grounded in human judgment rather than heavy reliance on large language models.
-
Knowledge of statistics and experience designing metrics, experiments, and quality assurance processes.
-
Strong communication skills, curiosity, and the ability to make progress independently in ambiguous, fast-moving environments.
Compensation & Benefits
Annual salary range: $100,000 to $170,000 USD. Visa sponsorship is available.
Location
On-site in Singapore. Fully remote independent contractor arrangements may be considered for candidates based elsewhere, particularly in Europe.