Staff Engineer – Enterprise AI Evaluation & Automation

Perplexity AI; New York (NY); On-site; USD 150,000 - 230,000; Full time

Job summary

Perplexity AI is seeking an experienced engineer to join the Enterprise Agent Evals & Efficiency team. You will own end-to-end evaluation environments, from building realistic scenarios to shipping improvements across the agent harness, tools, and skills.

Collaborate with AI Research, Connectors, and Platform teams to turn results into measurable gains in task success, latency, and cost. You will advance tool descriptions, contexts, and model routing strategies to ensure enterprise workflows are

Qualifications

• 5+ years of software or ML engineering with ownership of production systems
• Hands-on experience building/evaluating LLM-based applications or agents, including tool use and multi-step workflows
• Strong experimental and statistical judgment for meaningful task criteria and failure diagnosis
• Proficiency in a backend language such as Python, Go, or Rust

Responsibilities

• Build enterprise evaluation environments spanning connected applications, datasets, user roles, and permissions with multi-step workflows
• Own evaluation infrastructure, task suites, and automated graders calibrated against human judgment and outcomes
• Analyze agent trajectories and production failures to ship improvements across the harness, tools, and skills
• Develop reusable skills and improve tool descriptions, schemas, and result handling for better tool selection and error recovery
• Build model-selection and routing strategies balancing quality, latency, reliability, and cost

Skills

Python

Go

Rust

Software engineering

Machine learning engineering

Experimentation

Statistics

LLM/Agents

Tools

Python tooling

Production systems

Job description

Perplexity AI is seeking an experienced engineer to join the Enterprise Agent Evals & Efficiency team. You will own end-to-end evaluation environments, from building realistic scenarios to shipping improvements across the agent harness, tools, and skills.

Collaborate with AI Research, Connectors, and Platform teams to turn results into measurable gains in task success, latency, and cost. You will advance tool descriptions, contexts, and model routing strategies to ensure enterprise workflows are