Staff Engineer, Enterprise AI Evaluation & Automation

Fdetimes; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Perplexity is seeking an Enterprise Agent Evals & Efficiency Engineer in San Francisco. This hands-on role owns evaluation infrastructure, task suites, and automated graders to measure enterprise work, calibrate against human judgment, and drive end-to-end improvements across the agent harness, tools, and skills.

You will build realistic evaluation environments, analyze trajectories and failures, and ship tooling to improve model routing, context management, and cost per task, with attention to

Qualifications

• 5+ years of software engineering or ML engineering experience.
• Strong system design skills for scalable experimentation.
• Proficiency in a backend language (Python/Go/Rust) and multi-language stacks.
• Hands-on experience with LL M-based applications or agents.
• Experience with evaluation infrastructure, automated grading, and exploit of multi-step workflows.
• Leadership in ambiguous, cross-team initiatives from architecture to production.

Responsibilities

• Build enterprise evaluation environments with complex, multi-step workflows and reproducible state.
• Own evaluation infrastructure, task suites, and automated graders aligned with human judgment.
• Analyze agent trajectories and failures; ship improvements across harness, tools, and skills.
• Develop reusable skills and improve tool descriptions and result handling for reliable workflows.
• Build model selection and routing strategies to balance quality, latency, reliability, and cost.
• Improve context management, skill loading, caching, and execution strategies to reduce cost per task.
• Create regression gates and launch-readiness signals and validate production gains.

Skills

Backend development

System design

LLM engineering

Experimentation

Cross-team leadership

Python

Go

Rust

Tools

Python

Go

Rust

Job description

Perplexity is seeking an Enterprise Agent Evals & Efficiency Engineer in San Francisco. This hands-on role owns evaluation infrastructure, task suites, and automated graders to measure enterprise work, calibrate against human judgment, and drive end-to-end improvements across the agent harness, tools, and skills.

You will build realistic evaluation environments, analyze trajectories and failures, and ship tooling to improve model routing, context management, and cost per task, with attention to