Research Engineer, AI Data & Evaluations

Harvey; New York (NY); On-site; USD 230,000 - 450,000; Full time

Job summary

Harvey is building the future of professional services with frontier AI. We are hiring a Member of Technical Staff, Data & Eval to power post-training, benchmarking, and agent capability efforts.

You will own synthetic data pipelines, evaluation design, and the systems turning expert feedback into training signals. You’ll work with researchers and legal experts to turn failures into measurable improvements at scale.

Qualifications

• Experience building data pipelines and ML evaluation systems.
• Strong Python skills and ownership of production-quality code.
• Rigorous measurement with statistical reasoning and experimental design.

Responsibilities

• Build synthetic data pipelines for training and evaluation samples in complex legal reasoning.
• Design evaluations and verifiers that reflect real legal work, calibrate them against expert judgment, and investigate where they fail.
• Build and maintain capability benchmarks like our Legal Agent Benchmark (LAB), along with product-focused evals that track model performance.
• Work with legal domain experts to define quality metrics, rubrics, and grading mechanisms for training and evaluation.
• Own training and evaluation data quality, including provenance, coverage, and safeguards against contamination.
• Close the loop from product usage and customer feedback to new evaluations and training data, so models improve where customers need them most.
• Partner with researchers to investigate model failures and guide training and release decisions.

Skills

Python

ML evaluation

Experimental design

Data pipelines

Job description

Harvey is building the future of professional services with frontier AI. We are hiring a Member of Technical Staff, Data & Eval to power post-training, benchmarking, and agent capability efforts.

You will own synthetic data pipelines, evaluation design, and the systems turning expert feedback into training signals. You’ll work with researchers and legal experts to turn failures into measurable improvements at scale.