Research Engineer - Synthetic Data & Eval Systems

Fleet AI, Inc.

New York (NY)

On-site

USD 200,000 - 600,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Equity

Job summary

Fleet AI, Inc. is seeking a Research Engineer to build reusable and reliable systems for synthetic data and evaluations.

You will span data and task generation, verifiers, quality measurement, and the orchestration loop, developing new methods and turning the successful ones into scalable systems for researchers to reuse. You will work closely with internal research teams and frontier labs, study model behavior and traces, and build feedback loops between research findings and system

Qualifications

  • Strong engineering fundamentals and experience building reliable systems that others depend on.
  • Strong ML fundamentals. You understand how training objectives and optimization shape model behavior.
  • Ability to turn a research prototype into a maintainable system with clear interfaces and reliable execution.
  • Strong judgment about data and system quality: define good signals and diagnose failures.
  • Ability to work effectively with internal research teams, domain experts, and frontier lab partners.
  • High ownership, low ego, and preference for simple, boring systems.

Responsibilities

  • Build reusable systems for producing high-quality synthetic data, tasks, and verifiers across domains.
  • Define and enforce quality standards for each eval set and iterate until correct.
  • Build agentic hill-climbing and feedback loops to improve systems and outputs.
  • Collaborate with environment and research teams, frontier labs, and Fleet’s expert network to translate research needs into shared capabilities.
  • Own quality and usability end to end, including researcher education and adoption.

Skills

Python
Distributed systems
ML fundamentals
Production-grade systems
Debugging tools

Tools

Docker
Kubernetes
GPUs

Job description

Fleet AI, Inc. is seeking a Research Engineer to build reusable and reliable systems for synthetic data and evaluations.

You will span data and task generation, verifiers, quality measurement, and the orchestration loop, developing new methods and turning the successful ones into scalable systems for researchers to reuse. You will work closely with internal research teams and frontier labs, study model behavior and traces, and build feedback loops between research findings and system

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Synthetic Data & Evals
Synthetic Data & Evals

Fleet AI, Inc. • New York (NY)

On-site
USD 200,000 - 600,000
Equity
Staff Research Engineer, Frontier AI & Evaluation Systems
Staff Research Engineer, Frontier AI & Evaluation Systems

Dover • California

On-site
USD 180,000 - 240,000
Research Engineer — Synthetic Data & Task Pipeline
Research Engineer — Synthetic Data & Task Pipeline

Pragmatike • United States

Remote
USD 110,000 - 170,000
Research Engineer, Synthetic Data Pipelines
Research Engineer, Synthetic Data Pipelines

Pragmatike • United States

Hybrid
USD 180,000 - 240,000
Remote-friendly
Synthetic Data Research Engineer (Remote/Hybrid)
Synthetic Data Research Engineer (Remote/Hybrid)

Pragmatike • San Francisco (CA)

Hybrid
AUD 207,000 - 280,000
Research Engineer, AI Data & Evaluations
Research Engineer, AI Data & Evaluations

Harvey • New York (NY)

On-site
USD 230,000 - 450,000
Staff Engineer - Evaluation Infrastructure & Research Tools
Staff Engineer - Evaluation Infrastructure & Research Tools

Reflection • New York (NY)

On-site
USD 150,000 - 190,000
Top-tier compensation
Stock options
Health & wellness
+5
Staff AI Researcher: SFT, RL & Synthetic Data
Staff AI Researcher: SFT, RL & Synthetic Data

Ricursive Intelligence • Palo Alto (CA)

On-site
USD 210,000 - 320,000
Vision & Simulation AI: Synthetic Data Engineer
Vision & Simulation AI: Synthetic Data Engineer

Veriipro • Irving (TX)

On-site
USD 120,000 - 180,000
Applied AI Researcher: Environments & Tasks
Applied AI Researcher: Environments & Tasks

Fleet AI, Inc. • New York (NY)

On-site
USD 200,000 - 600,000
Equity