Software Engineering Director, Agentic Evaluations (Fully Remote)

Partner Company; United States; Remote; USD 260,000 - 320,000; Full time

Benefits offered by this job

Unlimited PTO

Equity

Parental leave

Job summary

Partner Company is seeking a seasoned backend/full-stack engineering leader to design and scale credible evaluations for live AI agents. You will own the evaluation platform's tech stack, from APIs to orchestration, and mentor teams across engineering and data science to build reusable evaluation primitives and proprietary benchmarks.

This fully remote role offers equity, unlimited PTO, and comprehensive parental leave.

Qualifications

• 10+ years of backend or full-stack development experience.
• 2+ years of engineering management experience.
• Hands-on experience evaluating AI agents and using frontier models.
• Total earnings of approximately $260,000-$320,000.
• Unlimited PTO, equity, and comprehensive parental leave.

Responsibilities

• Lead the development of credible, repeatable evaluations for live AI agents.
• Own the technology stack supporting the evaluation platform, from APIs to orchestration.
• Design reusable evaluation primitives applicable across software categories.
• Partner with data science teams to develop proprietary benchmarks.
• Promote the effective use of evaluations across agent-focused products.

Skills

Backend development

Full-stack development

Engineering management

AI agent evaluation

Tools

APIs

Orchestration

Job description

• Lead the development of credible, repeatable evaluations for live AI agents.
• Own the technology stack supporting the evaluation platform, from APIs to orchestration.

Key Responsibilities:

• Design reusable evaluation primitives applicable across software categories.
• Partner with data science teams to develop proprietary benchmarks.
• Promote the effective use of evaluations across agent-focused products.

Requirements:

• 10+ years of backend or full-stack development experience.
• 2+ years of engineering management experience.
• Hands-on experience evaluating AI agents and using frontier models.
• Total earnings of approximately $260,000-$320,000.
• Unlimited PTO, equity, and comprehensive parental leave.

Partner Company

This company focuses on building credible, scalable evaluations of AI agents from software vendors. It operates as a fully remote, inclusive team with a flexible culture and a focus on professional growth.