Lead Data Quality Engineer for AI & RL Systems

Wwshemi; United States; On-site; USD 130,000 - 225,000; Full time

Job summary

Wwshemi seeks a leader to head the data quality team at an AI/ML startup focused on reinforcement learning and agent evaluation. You will shape strategy and build systems to measure and improve training data for real-world tasks.

You will lead the development of scalable validation methods and dashboards, collaborate with researchers and data vendors, and mentor engineers to ensure rigorous, production-ready data workflows in a fast-moving startup environment.

Qualifications

• At least 5 years of research or engineering experience, including building AI or machine learning data evaluation systems.
• Experience leading technical projects or teams through ambiguous problems, from definition to implementation and iteration.
• Advanced proficiency with Python, Docker, and Linux.
• Background in AI research, evaluation, post-training, synthetic data, benchmarks, or data quality infrastructure.
• Strong understanding of what makes agent training data realistic, learnable, diverse, reliable, and useful.
• Experience designing metrics, experiments, and quality assurance processes, and translating research into production systems.
• Experience working with subject-matter experts and converting domain judgment into scalable review or data generation workflows.
• Clear written communication, strong attention to detail, and ability to work independently in an early-stage startup environment.

Responsibilities

• Lead development of systems that evaluate tasks across reinforcement learning environments, synthetic data, benchmarks, and specialized workflows.
• Define data quality strategy, establish standards, and design experiments to assess agent outputs.
• Develop scalable methods to validate synthetic data, including failure analysis, task mutation checks, and trajectory audits.
• Work with research engineers, domain experts, and data vendors to diagnose quality issues and improve data generation workflows.
• Turn research insights into production tools, dashboards, validation pipelines, and feedback loops.
• Develop shared standards for identifying training data that is useful beyond superficial correctness.
• Mentor research engineers and promote technical rigor, clear communication, and effective execution.

Skills

Python

Leadership

Research experience

Communication

Experiment design

Tools

Docker

Linux

Job description

Wwshemi seeks a leader to head the data quality team at an AI/ML startup focused on reinforcement learning and agent evaluation. You will shape strategy and build systems to measure and improve training data for real-world tasks.

You will lead the development of scalable validation methods and dashboards, collaborate with researchers and data vendors, and mentor engineers to ensure rigorous, production-ready data workflows in a fast-moving startup environment.