AI Training Data QC Automation Engineer

Invictus Direct; San Francisco (CA); On-site; USD 100,000 - 200,000; Full time

Benefits offered by this job

Relocation assistance

Visa support

Job summary

Invictus Direct is seeking a Research Engineer, QC Automation to build scalable quality-control systems for reinforcement-learning training data and evaluations. You will develop pipelines that judge data quality, design experiments, and set standards while operating in a startup environment that values curiosity about new domains.

The role requires proficiency in Python, Docker, and Linux, plus strong data-validation experience and the ability to execute independently.

Qualifications

• Proficiency in Python, Docker, and Linux environments.
• Strong judgment about what good data means and how to measure it.
• Genuine curiosity about unfamiliar domains and skill at asking the questions needed to understand them.
• Experience building scalable data-validation pipelines or automated QA/QC systems without a prescribed roadmap.
• Experience with benchmarks and evaluations.
• Early-stage startup experience and independent execution.

Responsibilities

• Create QC systems grounded in true understanding and human judgment without relying heavily on LLMs
• Define and enforce quality standards for training data
• Design experiments and metrics to grade agent outputs
• Partner with data vendors to diagnose agent failure modes, debug quality issues, and improve data-generation processes
• Build systems for auditing supplier datasets, including sampling strategies, rule-based and model-assisted validation pipelines, and feedback loops
• Integrate QC learnings into infrastructure tools and the data-vendor portal to reduce anomalies, inconsistencies, and edge cases

Skills

Python

Docker

Linux

Data quality judgment

QA/QC systems

Experiment design

Independent execution

Startup experience

Tools

Data validation pipelines

Model evaluation

Job description

Invictus Direct is seeking a Research Engineer, QC Automation to build scalable quality-control systems for reinforcement-learning training data and evaluations. You will develop pipelines that judge data quality, design experiments, and set standards while operating in a startup environment that values curiosity about new domains.

The role requires proficiency in Python, Docker, and Linux, plus strong data-validation experience and the ability to execute independently.