AI Training Data QC Automation Engineer

Invictus Direct; San Francisco (CA); In loco; USD 100.000 - 200.000; Tempo pieno

Vantaggi offerti da questo lavoro

Relocation assistance

Visa support

Descrizione del lavoro

Invictus Direct is seeking a Research Engineer, QC Automation to build scalable quality-control systems for reinforcement-learning training data and evaluations. You will develop pipelines that judge data quality, design experiments, and set standards while operating in a startup environment that values curiosity about new domains.

The role requires proficiency in Python, Docker, and Linux, plus strong data-validation experience and the ability to execute independently.

Competenze

• Proficiency in Python, Docker, and Linux environments.
• Strong judgment about what good data means and how to measure it.
• Genuine curiosity about unfamiliar domains and skill at asking the questions needed to understand them.
• Experience building scalable data-validation pipelines or automated QA/QC systems without a prescribed roadmap.
• Experience with benchmarks and evaluations.
• Early-stage startup experience and independent execution.

Mansioni

• Create QC systems grounded in true understanding and human judgment without relying heavily on LLMs
• Define and enforce quality standards for training data
• Design experiments and metrics to grade agent outputs
• Partner with data vendors to diagnose agent failure modes, debug quality issues, and improve data-generation processes
• Build systems for auditing supplier datasets, including sampling strategies, rule-based and model-assisted validation pipelines, and feedback loops
• Integrate QC learnings into infrastructure tools and the data-vendor portal to reduce anomalies, inconsistencies, and edge cases

Conoscenze

Python

Docker

Linux

Data quality judgment

QA/QC systems

Experiment design

Independent execution

Startup experience

Strumenti

Data validation pipelines

Model evaluation

Descrizione del lavoro

Invictus Direct is seeking a Research Engineer, QC Automation to build scalable quality-control systems for reinforcement-learning training data and evaluations. You will develop pipelines that judge data quality, design experiments, and set standards while operating in a startup environment that values curiosity about new domains.

The role requires proficiency in Python, Docker, and Linux, plus strong data-validation experience and the ability to execute independently.