Agentic Reliability Engineer (ARE)

Slalom – Agentic Reliability Engineer (ARE)

Slalom · Agentic Managed Services · Remote (US) with possible client travel · Principal or Senior Principal level

ABOUT THE ROLE

AI agents are moving into production faster than most organizations can operate them reliably and closing that gap is exactly why Slalom built its Agentic Managed Services (AMS) practice. As an Agentic Reliability Engineer, you are the technical heart of that work: the engineer who makes AI agents dependable in the real world.

Where a traditional Site Reliability Engineer keeps software services healthy, you own the reliability of AI agents. These are systems that reason, make decisions, and act on a client's behalf, but can fail in ways classical engineering never had to consider. You'll help design and stand up agentic solutions with clients, prove they hold up under real conditions, and then keep them performing in production: watching how they behave, diagnosing why they drift, tuning and evaluating them, and safely rolling out changes as they evolve.

Your day to day will look like a small senior delivery pod working shoulder-to-shoulder with the client and with Slalom's platform team. It's a collaborative, high-ownership environment: you are the final technical authority on how the agents behave, and the person a client trusts to tell them their systems are reliable, well-governed, and getting better over time. Because our agentic reliability engineering group is a new and fast-growing practice, you'll also help shape how the work is done. We expect you to provide input in to the patterns, tooling, and standards that make running production agents repeatable.

This is a role for engineers who want to be at the frontier of what AI systems actually do in production. You thrive on work with real stakeholders, measured outcomes, and sustained impacts.

WHAT YOU'LL DO

• Design & deploy agentic solutions

• Partner with clients to shape the opportunity and design the agent workflow — what the agents should do, how their performance will be judged, and what guardrails keep them safe.

• Build the first working agents: stand up the initial workflow, evaluation harness, and guardrails, and demonstrate that the solution meets the client's acceptance criteria under real conditions.

• Define the reliability baseline the solution must hold in production including the SLOs, evaluation criteria, and autonomy posture that govern it.

• Run & continuously improve production agents

• Own root-cause analysis and ongoing performance of the agent workflow itself: why it behaves as it does, and how to make it measurably better over time.

• Own the evaluation harness: configure, run, and interpret agent evaluations against workflow-specific acceptance criteria.

• Monitor drift signals (e.g. output-quality degradation, latency trends, evaluation pass-rate decline, cost anomalies) and remediate before they breach SLOs.

• Run A/B and shadow evaluations when agent behavior changes are proposed

• Capture operational findings, incident lessons, and evaluation-calibration notes so the pod's knowledge compounds over time rather than resetting.

• Measure, analyze & communicate performance

• Define and track the KPIs that describe how well the agents are performing and investigate when they move in unexpected directions.

• Translate telemetry into narratives the team and client can act on.

• Develop and communicate clear plans to improve agent performance and reporting on their impact once shipped.

• Serve as the final technical authority for AI agent incidents including owning root-cause analysis of the agent workflow, remediation, and postmortem documentation.

• Determine whether an incident reflects a transient error, agent drift, workflow-logic failure, or an underlying model-behavior change.

• Safe change & controlled rollout

• Manage change control for agent workflow updates: version the change, run the evaluation suite, document the expected behavioral delta, and propose the go-live schedule.

• Stage rollouts with shadow-deployment patterns before promoting to full release; monitoring the stability window and rolling back per runbook if thresholds are breached.

• Manage identity, access, and integrations: app registrations and credentials, access control and permissions, and the authentication, authorization, and contracts that govern how agents communicate with other tools and systems.

• Manage cost and infrastructure: right-sizing the compute agents run on, setting token and budget caps per agent, optimizing workflow steps for efficient resource use, and reporting spend — and the cost impact of the changes you make — to the team and client.

• Build reusable tooling & help shape the practice

• Contributing reusable agents, evaluation templates, and runbook patterns so each new engagement starts further ahead than the last.

• Acting as a primary production signal for Slalom's AMS platform team — surfacing the gaps whose closure most improves how the work gets done.

• Helping define the standards and playbooks for a new and fast-growing engineering discipline.

• CORE SKILLS

• Deep analytical & debugging rigor — defines what “correct” looks like for an agent decision and builds the test to prove it.

• Critical thinking — approaches problems with a problem solving mindset.

• Communication & translation — turns telemetry and metrics into clear narratives and reports a mixed team-and-client audience can act on; can explain why an agent is behaving as it is, and what the plan is to change it.

• Solutioning & client fluency — can shape a solution with a client, translate a business problem into an agentic workflow design, and make the value case.

• Workflow root-cause & performance ownership — owns RCA of the agent workflow itself and drives its improvement.

• Production engineering — strong Python/TypeScript; reads and modifies agent orchestration code.

• AI / agent reliability — evaluations, drift detection, controlled rollout, observability.

• A builder's instinct for reuse — reaches for the reusable pattern over the one-off.

• WHAT YOU BRING

• Required

• 5+ years of engineering experience in a production environment (software engineering, data engineering, platform/DevOps, or SRE).

• Strong Python or TypeScript; ability to read, modify, and evaluate agent orchestration code without extensive scaffolding.

• Hands-on experience with at least one production AI/ML system (model serving, pipeline orchestration, or an LLM-based application).

• Familiarity with observability tooling (distributed tracing, structured logging, alerting) and incident-response workflows.

• Analytical rigor: comfortable defining what correct looks like for an AI agent decision, and building a test to verify it.

• Preferred

• Client-facing or solutioning experience — the ability to scope and shape technical engagements with clients, not only operate them.

• Direct experience with LLM-based agent frameworks — LangChain, LangGraph, CrewAI, AutoGen, AWS Bedrock Agents, or equivalent.

• Evaluation frameworks for AI systems (Langfuse, Braintrust, RAGAS, DeepEval, or custom harnesses).

• Familiarity with agentic memory architectures — episodic, semantic, and procedural stores; vector databases; RAG.

• AI governance concepts: safety guardrails, NIST AI RMF, or OWASP LLM Top 10.

• WHAT SUCCESS LOOKS LIKE

• Agentic system stability & efficiency — the agents you own run reliably and improve over time: strong evaluation pass rates, drift caught early, fast recovery when incidents happen, clean rollouts, and steadily better workflow performance and cost efficiency.

• Client outcomes — your clients trust that their agentic systems are dependable, well-governed, and advancing, reflected in SLO attainment and the growth of the outcomes the agents are responsible for.

• Revenue impacted — unlike a traditional services role, you are measured on the value you drive, not hours booked.

• Feedback, collaboration & partnership — you make the people around you better: giving and acting on candid feedback, sharing what you learn, and partnering effectively with Slalom teammates across capabilities, platform, and client teams to move the work forward together.

• WHY THIS ROLE, WHY NOW

• AI agents are moving from novelty to critical infrastructure, and the engineers who build the discipline of running them reliably will define the field for the next decade. Being able to say “I have run agents in production, at SLA, and debugged them”, is one of the most exciting and evolving skillsets engineering today. This role puts you at the center of it: real clients, real outcomes, and a platform team investing in the tooling that makes the work tractable, at a fir