Research Engineer QC Automation

Dreams 2 Reality Recruitment; San Francisco (CA); On-site; USD 100,000 - 200,000; Full time

Benefits offered by this job

Relocation support

Visa sponsorship

Job summary

HUD builds infrastructure for companies creating training data for AI agents. As a Research Engineer, QC Automation, you’ll develop automated quality-control systems grounded in human judgment to ensure data quality and scalability, with limited reliance on LLM-based evaluation.

This highly autonomous role requires turning unclear quality requirements into measurable, scalable processes, learning quickly, and taking ownership in a fast-moving environment.

Qualifications

• Strong proficiency in Python, Docker, and Linux environments.
• Excellent judgement about what constitutes good data and how to measure it.
• Genuine curiosity about unfamiliar domains, with the ability to ask the right questions to develop a deep understanding quickly.
• Experience building scalable data-validation pipelines, automated QA/QC systems, or similar infrastructure.
• Experience designing or working with benchmarks and evaluations.
• Comfort working in an early-stage startup environment, taking ownership and executing independently.
• A track record of learning quickly and tackling problems that don’t come with clear instructions.

Responsibilities

• Build quality-control systems grounded in human judgment and a deep understanding of data quality, with limited reliance on LLM-based evaluation.
• Define, formalize, and enforce quality standards for training data.
• Design experiments, benchmarks, and metrics to evaluate agent performance.
• Partner with data vendors to identify agent failure modes, debug quality issues, and improve data-generation processes.
• Build systems for auditing supplier datasets, including sampling strategies, rule-based validation, model-assisted validation, and feedback loops.
• Integrate QC insights into HUD’s infrastructure and data-vendor portal to reduce anomalies, inconsistencies, and edge cases.
• Work across unfamiliar domains and turn loosely defined quality problems into reliable, repeatable processes.

Skills

Python

Docker

Linux

Data quality

QA pipelines

Benchmarks

Startup environment

Job description

Research Engineer, QC Automation

Location: San Francisco Bay Area, CA — On-site
Employment: Full-time
Compensation: $100,000–$200,000
Visa: Relocation and visa support available for strong candidates
Experience: Technical aptitude and learning potential matter more than years of experience.

About the Role

HUD builds infrastructure for companies creating training data for AI agents. As demand grows, we need robust systems that can maintain and scale data quality.

As a Research Engineer, QC Automation , you’ll build the systems that make that possible. You’ll develop automated quality-control infrastructure grounded in human judgment and a deep understanding of what makes training data useful—not simply by relying on LLMs to judge other LLMs.

This is a highly autonomous role for someone who enjoys ambiguous problems, learns quickly, and can turn unclear quality requirements into measurable, scalable systems.

What You’ll Do

• Build quality-control systems grounded in human judgment and a deep understanding of data quality, with limited reliance on LLM-based evaluation.
• Define, formalize, and enforce quality standards for training data.
• Design experiments, benchmarks, and metrics to evaluate agent performance.
• Partner with data vendors to identify agent failure modes , debug quality issues, and improve data-generation processes.
• Build systems for auditing supplier datasets , including sampling strategies, rule-based validation, model-assisted validation, and feedback loops.
• Integrate QC insights into HUD’s infrastructure and data-vendor portal to reduce anomalies, inconsistencies, and edge cases.
• Work across unfamiliar domains and turn loosely defined quality problems into reliable, repeatable processes.

What We’re Looking For

• Strong proficiency in Python, Docker, and Linux environments.
• Excellent judgment about what constitutes good data and how to measure it.
• Genuine curiosity about unfamiliar domains, with the ability to ask the right questions to develop a deep understanding quickly.
• Experience building scalable data-validation pipelines, automated QA/QC systems, or similar infrastructure without a prescribed roadmap.
• Experience designing or working with benchmarks and evaluations.
• Comfort working in an early-stage startup environment , taking ownership and executing independently.
• A track record of learning quickly and tackling problems that don#39;t come with clear instructions.

Strong Signals

• Knowledge of statistics and experimental design.
• Strong written and verbal communication skills.
• Comfort designing metrics, experiments, and QA/QC processes.
• Ability to construct tasks for new evaluations and benchmarks.
• Comfort operating in unstructured problem spaces.
• A tendency to dig into the underlying problem rather than reaching for the first available tool or solution.