Staff Engineer - Evaluation Infrastructure & Research Tools

Reflection; New York (NY); On-site; USD 150,000 - 190,000; Full time

Benefits offered by this job

Top-tier compensation

Stock options

Health & wellness

Meals

Parental leave

Unlimited PTO

Visa sponsorship

Team building

Job summary

Reflection is seeking an engineer to build evaluation infrastructure used by researchers to understand model capabilities, guide training decisions, and assess readiness for release.

You will design APIs, build scalable execution systems, optimize performance, ensure reproducibility, and deliver researcher-friendly tools across teams. Join a mission-driven lab building open foundations for AI.

Qualifications

• Strong proficiency in Python, C++, or Rust with attention to concurrency and performance.
• Experience designing and operating reliable distributed systems with clear interfaces and failure modes.
• Knowledge of model inference and evaluation and how configurations affect quality.
• Ability to investigate complex problems across system boundaries and identify root causes.

Responsibilities

• Design and evolve evaluation abstractions with APIs and data models for tasks, datasets, configurations, model outputs, and scores.
• Build scalable, reliable orchestration and execution infrastructure for evaluating model checkpoints.
• Profile and optimize inference servers and distributed execution to improve throughput and time-to-results.
• Establish evaluation correctness and reproducibility with tests, versioning, and provenance.

Skills

Python

C++

Rust

Job description

Reflection is seeking an engineer to build evaluation infrastructure used by researchers to understand model capabilities, guide training decisions, and assess readiness for release.

You will design APIs, build scalable execution systems, optimize performance, ensure reproducibility, and deliver researcher-friendly tools across teams. Join a mission-driven lab building open foundations for AI.