About this role
About the Role
Join the founding team at a Y Combinator-backed AI assistant startup to build the models and systems behind a useful, reliable AI agent. You will lead hands-on research and engineering that improves answer quality while managing latency, cost, and safety, and ship improvements to users regularly.
What You'll Do
-
Build evaluation harnesses using model judges, human preference data, and regression tests to guide release decisions.
-
Post-train models through supervised fine-tuning, preference tuning, and distillation, then ship the results to production.
-
Develop model routing that balances answer quality, latency, and cost across smaller and frontier models.
-
Improve agent performance on websites without APIs, including navigation, sign-in, and checkout, with appropriate escalation when needed.
-
Identify and address prompt injection, data leakage, and other risks through adversarial testing and evaluations.
-
Advance memory, retrieval, tool selection, and decisions about when the agent should act or ask for help.
-
Review real conversations and turn findings into evaluation and model improvements.
What We're Looking For
-
At least 4 years of ML or software engineering experience at an AI lab, AI product company, or similar organization.
-
Hands-on experience fine-tuning or post-training LLMs and deploying the results to production.
-
Experience designing evaluations that teams have used to make product release decisions.
-
Background in at least one area: LLM red teaming, browser or computer-use agents, model routing, or production agents with tools and persistent memory.
-
Strong Python skills and experience with ML training and evaluation tools, plus TypeScript experience in a product codebase.
-
A hands-on, founding-team approach, with interest in setting technical direction and helping grow the team.
Compensation & Benefits
Salary range: $200,000 to $300,000 USD annually. Visa sponsorship is not available.
Location
On-site in San Francisco, California, United States.