Engineering Manager, AI Evaluation Systems

Cervin; Northern (KY); Hybrid; USD 163,000 - 224,000; Full time

Benefits offered by this job

RSUs

Health insurance

Vision insurance

Dental insurance

Mental health benefits

Job summary

LaunchDarkly is seeking an Engineering Manager for the AgentControl Evaluations team. You will lead a group building offline evaluations, playgrounds, LLM-as-judges, and annotated datasets, shaping the core product's evaluation stack.

You'll partner with product and design to scope work, own delivery, and drive a culture of high performance. Expect collaboration across AI, observability, and feature management teams, with Go, Python, and Typescript at the core.

Qualifications

• 3+ years of software engineering experience
• 2+ years managing a backend or infrastructure team
• Ability to translate business goals into engineering plans with reliable estimations
• Strong communication skills in distributed, cross-timezone teams
• Experience coaching and developing engineers, including performance management
• Familiarity with observability practices (metrics, tracing, alerting, logging)
• Familiarity with AI providers (Anthropic, OpenAI, Bedrock, Gemini) and AI tooling (Langchain, Arize)
• Experience with Go and Python or similar backend languages (Java, Rust, Ruby)

Responsibilities

• Lead and develop a team of engineers with coaching and career growth.
• Partner with PM and Design to scope and sequence roadmap items with reliable estimates.
• Own delivery of features within timelines and quality requirements.
• Collaborate with adjacent teams on infrastructure decisions and cross-team dependencies.
• Communicate roadmaps, risks, and progress to engineering leadership and stakeholders.
• Establish team norms to promote collaboration and reduce silos.
• Improve reliability, observability, and incident response for production systems.
• Participate in hiring to grow the team and raise engineering bar.

Skills

Team leadership

Roadmap collaboration

Observability

AI tooling familiarity

Go & Python

Tools

Go

Python

Typescript

Job description

LaunchDarkly is seeking an Engineering Manager for the AgentControl Evaluations team. You will lead a group building offline evaluations, playgrounds, LLM-as-judges, and annotated datasets, shaping the core product's evaluation stack.

You'll partner with product and design to scope work, own delivery, and drive a culture of high performance. Expect collaboration across AI, observability, and feature management teams, with Go, Python, and Typescript at the core.