Research Engineer - Scalable QC for RL Training Data

HUD; San Francisco (CA); On-site; USD 140,000 - 200,000; Full time

Benefits offered by this job

Medical, dental, vision coverage (Blue

Lunch & dinner in office

Holiday break & PTO/holidays

Equinox membership

401k

Commuter benefits

Unlimited AI tool tokens

Job summary

HUD is seeking Research Engineers to automate QC for training data generated through our ML infrastructure, building systems that scale quality for RL training. You’ll implement robust data validation pipelines, sampling strategies, and feedback loops while collaborating with data vendors.

Ideal candidates have strong Python, Docker, and Linux skills, startup experience, and a knack for designing metrics and experiments to assess data quality and task realism.

Qualifications

• Experience building scalable QA/QC pipelines for datasets.
• Ability to define quality standards for training data.
• Experience with benchmarks and RL evals.
• Startup environment and independent work ability.
• Strong communication across time zones.

Responsibilities

• Create QC systems based on true understanding and human judgement, without relying heavily on LLMs.
• Define and enforce quality standards for training data.
• Design experiments and metrics to grade agent outputs.
• Partner with data vendors to debug quality issues and diagnose agent failure modes, provide actionable feedback, and improve their data generation processes
• Translate QC learnings into systems for auditing supplier-generated datasets, including sampling strategies, validation pipelines (rule-based and model-assisted), and feedback loops
• Continuously integrate QC learnings into infrastructure tools and data vendor portal to reduce anomalies, inconsistencies, and edge cases

Skills

Python

Docker

Linux

Quality Assurance

RL benchmarks

Education

Bachelor's degree in CS/CE/Math

Tools

Git

Airflow

CI/CD

Job description

HUD is seeking Research Engineers to automate QC for training data generated through our ML infrastructure, building systems that scale quality for RL training. You’ll implement robust data validation pipelines, sampling strategies, and feedback loops while collaborating with data vendors.

Ideal candidates have strong Python, Docker, and Linux skills, startup experience, and a knack for designing metrics and experiments to assess data quality and task realism.