Lead Research Engineer, Data Quality (Visa Sponsorship - United States)

IllTip

United States

On-site

USD 180,000 - 280,000

Full time

7 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

IllTip is seeking a leadership research engineering head to drive a group focused on measuring and improving data quality for AI training and evaluation. You will oversee reinforcement learning environments, synthetic datasets, benchmarks, and domain-specific workflows, setting the quality strategy and the systems behind it.

You will lead a team of researchers and engineers, turning research findings into production tools, dashboards, and validation pipelines, while mentoring staff and

Qualifications

  • Minimum 5 years of research or engineering experience in AI data evaluation and quality systems.
  • Experience leading technical teams or projects from problem definition through implementation.
  • Advanced proficiency in Python, Docker and Linux environments.
  • Strong background in AI evaluations, post-training, synthetic data validation, benchmarks or related evaluation infrastructure.
  • Clear written communication and startup environment experience.

Responsibilities

  • Lead a team that builds systems to evaluate large volumes of tasks across environments, datasets, benchmarks and specialized workflows.
  • Define data quality strategy by building quality-control systems, setting standards and designing experiments to assess agent outputs.
  • Develop scalable methods to validate synthetic data, including failure-mode analysis, task mutation checks and trajectory auditing.
  • Work with research engineers, domain experts and data vendors to diagnose quality problems and improve data generation.
  • Turn research findings into production tools, dashboards, validation pipelines and feedback loops.
  • Set rigorous standards for judging whether agent training data is useful, and mentor research engineers.

Skills

Python
Clear written communication
Startup experience

Tools

Docker
Linux

Job description

This is a leadership role at an early-stage AI company that builds training data for AI agents. You will lead a research engineering group focused on measuring and improving the quality of data used to train and evaluate agents, including reinforcement learning environments, synthetic datasets, benchmarks and domain-specific workflows. You will set the quality strategy and the systems behind it, and work closely with researchers, domain experts and data vendors.

Responsibilities
  • Lead a team that builds systems to evaluate large volumes of tasks across environments, datasets, benchmarks and specialized workflows.
  • Define data quality strategy by building quality-control systems, setting standards and designing experiments to assess agent outputs.
  • Develop scalable methods to validate synthetic data, including failure-mode analysis, task mutation checks and trajectory auditing.
  • Work with research engineers, domain experts and data vendors to diagnose quality problems and improve data generation.
  • Turn research findings into production tools, dashboards, validation pipelines and feedback loops.
  • Set rigorous standards for judging whether agent training data is useful, and mentor research engineers.
Requirements
  • At least 5 years of research or engineering experience building AI or machine learning data evaluation and quality systems.
  • Prior experience leading technical teams or projects from problem definition through implementation and iteration.
  • Advanced proficiency in Python, Docker and Linux environments.
  • Strong background in AI evaluations, post-training, synthetic data validation, benchmarks or related evaluation infrastructure.
  • Clear written communication, careful attention to detail, and experience working in an early-stage startup environment.
  • Ability to work with subject-matter experts and convert their judgment into scalable review or data generation systems.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Founding Engineer, Environments and Data (Visa Sponsorship - San Francisco, United States)
Founding Engineer, Environments and Data (Visa Sponsorship - San Francisco, United States)

IllTip • San Francisco (CA)

On-site
USD 180,000 - 280,000
Lead Research Engineer, Data Quality
Lead Research Engineer, Data Quality

HUD • San Francisco (CA)

Hybrid
USD 180,000 - 260,000
Top-tier medical, dental, vision
Lunch & dinner in office
Holiday break
+4
Full-Stack Software Engineer, Reinforcement Learning (Visa Sponsorship - San Francisco, United States)
Full-Stack Software Engineer, Reinforcement Learning (Visa Sponsorship - San Francisco, United States)

IllTip • San Francisco (CA)

On-site
USD 140,000 - 190,000
Applied Scientist / Research Engineer — Data Quality & Evaluation
Applied Scientist / Research Engineer — Data Quality & Evaluation

Maxinsights • Santa Clara (CA)

On-site
USD 140,000 - 210,000
Health insurance
Vision care
Dental coverage
+4
Quality Lead, Gen AI
Quality Lead, Gen AI

Scale AI, Inc. • San Francisco (CA)

On-site
USD 134,000 - 168,000
Quality Lead, Gen AI
Quality Lead, Gen AI

Scale AI, Inc. • New York (NY)

On-site
USD 134,000 - 168,000
Health, dental and vision coverage
Retirement benefits
Learning and development stipend
+2
Quality Operations Lead, Gen AI
Quality Operations Lead, Gen AI

Scale AI, Inc. • New York (NY)

On-site
USD 134,000 - 168,000
Artificial Intelligence Researcher - Post Training
Artificial Intelligence Researcher - Post Training

Velocity Tech • San Francisco (CA)

On-site
USD 120,000 - 180,000
Quality Operations Lead, Gen AI
Quality Operations Lead, Gen AI

Scale AI, Inc. • San Francisco (CA)

On-site
USD 134,000 - 168,000
Health, dental & vision coverage
Retirement benefits
Learning & development stipend
+2
Quality Lead, Gen AI
Quality Lead, Gen AI

Scaleai • San Francisco (CA), New York (NY)

On-site
USD 134,000 - 168,000
Health, dental and vision coverage
Retirement benefits
Learning and development stipend
+2