Senior Agentic AI Operations & Reliability Engineer

MultiPlan; McLean (VA); On-site; USD 165,000 - 185,000; Full time

Benefits offered by this job

Health insurance

401k

Bonus opportunity

Job summary

Claritev is seeking a Principal Agentic AI Operations Engineer to lead the operationalization, evaluation, and reliable production performance of advanced agentic AI systems for healthcare products.

You will define what good looks like for agent behavior in high-stakes workflows, build evaluation harnesses, benchmarks, and tooling to prove reliability and safety at scale in production environments.

Qualifications

• 10+ years of hands-on experience in software engineering, ML engineering, platform/SRE engineering, or a related technical discipline.

Responsibilities

• Own the end-to-end operational lifecycle of production agentic AI systems: deployment, versioning, rollout strategies, monitoring, incident response, rollback, and continuous improvement.

Skills

Python proficiency

Distributed systems design

MLOps/LLMOps concepts

Cloud computing

Communication

Problem solving

Education

Bachelor's degree in Computer Science, Engineering, Data Science, a quantitative discipline, or a related field

Master's degree or PhD preferred

Tools

Terraform

Kubernetes

LangSmith

Langfuse

Arize Phoenix

Ragas

DeepEval

promptfoo

OpenAI Evals

Job description

Claritev is seeking a Principal Agentic AI Operations Engineer to lead the operationalization, evaluation, and reliable production performance of advanced agentic AI systems for healthcare products.

You will define what good looks like for agent behavior in high-stakes workflows, build evaluation harnesses, benchmarks, and tooling to prove reliability and safety at scale in production environments.