About this role
OpenTrain AI
The work
- Own research and evaluation initiatives from problem framing through data design, quality calibration, and signal validation.
- Design machine learning data systems with task definitions, annotation schemas, rubrics, incentives, and pipelines tied to model performance.
- Analyze model and system failures to find root causes, edge cases, and improvement opportunities.
- Turn unclear system behavior into evaluation frameworks and new data categories.
- Work with researchers and domain experts to calibrate quality and improve evaluation signals.
- Iterate on evaluations, datasets, and feedback loops to improve model or agent performance.
- Make evidence-based quality decisions when data integrity or signal strength is not sufficient.
- Find gaps in evaluation coverage and recommend where to invest, iterate, or stop.
- Collaborate remotely with researchers, domain experts, operators, and cross-functional teams.
Requirements
-
Language: Fluent in English.
-
Required skills: Experience designing machine learning datasets, evaluation frameworks, and quality assurance processes; strong judgment about research signal quality and evidence readiness; the ability to structure messy, real-world system behavior into research and evaluation opportunities, explain tradeoffs and limitations clearly, work directly with experts during calibration and iteration, and use a systems-level view of end-to-end model or agent performance.
-
Skills: Research Signal Judgment, ML-Oriented Data Design, Ops-to-Research Translation, RL Environments
-
Open to applicants in
-
Algeria
-
Argentina
-
Austria
-
Bahrain
-
Bangladesh
-
Belgium
-
Brazil
-
Canada
-
Chile
-
Colombia
-
Denmark
-
Egypt
-
France
-
Germany
-
Ghana
-
Greece
-
India
-
Indonesia
-
Italy
-
Japan
-
Jordan
-
Kazakhstan
-
Kenya
-
Kuwait
-
Kyrgyzstan
-
Lebanon
-
Malaysia
-
Mexico
-
Morocco
-
Netherlands
-
Nigeria
-
Oman
-
Pakistan
-
Palestine
-
Peru
-
Philippines
-
Portugal
-
Qatar
-
Saudi Arabia
-
Singapore
-
South Africa
-
Spain
-
Sri Lanka
-
Switzerland
-
Taiwan
-
Thailand
-
Tunisia
-
United Arab Emirates
-
United Kingdom
-
United States
-
Uzbekistan
-
Vietnam
-
Preferred
-
Helpful background: Reinforcement learning environments, simulators, feedback-driven training systems, agentic systems, applied research, production systems, complex-task evaluation, or expert incentive design.