Staff Software Engineer - Agent Evaluation & Efficiency

Wwshemi; New York (NY); On-site; USD 180,000 - 240,000; Full time

Job summary

Perplexity is seeking an experienced software/ML engineer to own enterprise evaluation environments, spanning connected systems, datasets, user roles, and permissions. You will build and maintain automated graders, analyze production failures, and ship improvements across the agent harness, tools, and skills to boost task success, latency, and cost efficiency.

You will collaborate with AI Research, Connectors, and Platform teams to turn experimental results into measurable production gains,

Qualifications

• 5+ years of software engineering or machine learning engineering experience in production systems.
• Strong backend design skills for scalable experimentation and data-intensive apps.
• Hands-on experience with LLM-based applications, agents, and multi-step workflows.
• Ability to work across a multi-language production stack and ship reliable results.

Responsibilities

• Build enterprise evaluation environments with complex, multi-step workflows.
• Own evaluation infrastructure, task suites, and automated graders calibrated to human judgment.
• Analyze agent trajectories and failures; ship improvements to tooling and skills.
• Develop reusable skills and improve tool descriptions, schemas, and result handling.

Skills

Python

Go

Rust

LLM apps

multi-language stack

Job description

Perplexity is seeking an experienced software/ML engineer to own enterprise evaluation environments, spanning connected systems, datasets, user roles, and permissions. You will build and maintain automated graders, analyze production failures, and ship improvements across the agent harness, tools, and skills to boost task success, latency, and cost efficiency.

You will collaborate with AI Research, Connectors, and Platform teams to turn experimental results into measurable production gains,