About this role
About the Role
Lead the development of agent intelligence that helps mechanical engineers complete complex, multi-step workflows across desktop engineering software. As a hands-on technical lead on a small AI engineering team, you will shape the agent architecture, connect user research with product development, and improve the reliability and cost efficiency of real-world workflows.
What You'll Do
-
Own agent task success metrics, establish performance baselines, and systematically improve completion rates.
-
Build rigorous, reproducible evaluation infrastructure grounded in validated user stories and real engineering workflows.
-
Set per-task token budgets and track the cost of completed workflows.
-
Work with researchers and engineering domain experts to map, interview about, and validate user workflows.
-
Translate validated user stories into testable evaluations and prioritize workflow coverage by customer value and technical feasibility.
-
Make architecture decisions across tool calling, state management, error recovery, model routing, and context management.
-
Set technical direction for a small team, review designs and code, unblock teammates, and contribute production code.
-
Collaborate with product, integrations, and customers to align agent behavior with real-world use.
What We're Looking For
-
At least 7 years of software engineering experience, including 2 or more years building and shipping LLM-based agents that take real-world actions.
-
Deep experience with LLM application architecture, including model selection, context management, retrieval, tool calling, and orchestration.
-
Strong Python skills and familiarity with function calling, tool APIs, tracing or observability, and evaluation tooling.
-
Experience building benchmarks for task completion, cost efficiency, and failure analysis.
-
Hands-on technical leadership and code review experience on a small engineering team.
-
Experience applying AI or LLM tooling to proprietary engineering data or desktop engineering software, with background in mechanical engineering, CAD, CAE, PLM, or a related domain.
-
Familiarity with desktop automation and enterprise workstation constraints is valuable.
Compensation & Benefits
Salary range: $160,000 to $250,000 USD annually. Visa sponsorship is not available.
Location
On-site in San Francisco, California, United States.