About this role
About the Role
Lead the development of agent intelligence that helps engineers complete complex workflows across desktop engineering software. You will be a hands-on technical lead for a small AI team, partnering with user research and domain experts to deliver reliable, cost-efficient capabilities grounded in real engineering workflows.
What You'll Do
-
Own agent architecture for tool calling, state management, error recovery, model routing, and context management.
-
Define and improve task success metrics, establishing baselines and tracking workflow completion, cost, and failure modes.
-
Build rigorous, reproducible evaluation infrastructure that translates validated user stories into test cases.
-
Work with researchers, domain experts, and engineers to map user workflows and prioritize valuable agent capabilities.
-
Set per-task token budgets and measure cost per completed workflow.
-
Lead a small team through technical direction, design reviews, code reviews, and hands-on implementation.
-
Collaborate with product, integrations, and customers to align agent behavior with real-world use.
What We're Looking For
-
At least 7 years of software engineering experience, including 2 or more years building LLM agents that take real-world actions.
-
Deep knowledge of LLM application architecture, including model selection, context management, retrieval, tool calling, and orchestration.
-
Experience creating agent evaluation and benchmarking frameworks for task completion, cost efficiency, and failure analysis.
-
Strong Python skills and familiarity with function calling, tool APIs, observability or tracing, and evaluation tools.
-
Hands-on technical leadership experience guiding a small engineering team while continuing to write production code.
-
Experience shipping AI or LLM tools over proprietary engineering data or desktop engineering software.
-
Experience with desktop automation or application APIs, such as COM, and familiarity with enterprise workstation constraints.
-
Background in mechanical engineering software, CAD/CAE, PLM, or a related engineering domain.
Compensation & Benefits
Base salary: $160,000 to $250,000 annually. Visa sponsorship is not available.
Location
On-site in San Francisco, California, United States.