Staff Engineer, AI Evaluation & Data Systems

Perplexity AI; New York (NY); On-site; USD 120,000 - 190,000; Full time

Job summary

Perplexity AI invites you to join as an Answer Quality Engineer to help design and implement evaluation systems for high-quality answers. You will work across prompts, tools, search, datasets, and models to improve user experiences by identifying and solving answer-quality issues through data-driven evaluations.

You will collaborate with data scientists, engineers, and product teams to turn evaluation findings into concrete product improvements, building scalable, observable infrastructure for

Qualifications

• 4+ years of software, data, or ML engineering experience shipping and operating production systems.
• Strong proficiency in Python and SQL with fundamentals in system design and data modeling.
• Experience with big-data pipelines and distributed systems.

Responsibilities

• Build shared evaluation infrastructure that helps teams run reliable evals, analyze results, and make product and model decisions
• Develop the platform for replaying and analyzing agent traces to reproduce production behavior and diagnose failures
• Build and operate scalable systems for processing, storing, and monitoring interaction, trace, and evaluation data
• Partner with data scientists, engineers, and product teams to turn answer-quality problems into evaluations, analyses, and product improvements
• Operate in a small, high-impact team where your work directly shapes how Perplexity measures and improves Answer Quality

Skills

Python

SQL

System design

Data modeling

Distributed systems

Tools

Databricks

Snowflake

ClickHouse

Job description

Perplexity AI invites you to join as an Answer Quality Engineer to help design and implement evaluation systems for high-quality answers. You will work across prompts, tools, search, datasets, and models to improve user experiences by identifying and solving answer-quality issues through data-driven evaluations.

You will collaborate with data scientists, engineers, and product teams to turn evaluation findings into concrete product improvements, building scalable, observable infrastructure for