Research Manager: Scalable AI Training Data & Evals

HUD; San Francisco (CA); Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Competitive compensation

Medical/dental/vision coverage

Lunch and dinner when in office

Holiday break

401k

Commuter benefits

Equinox membership

Unlimited AI tokens

Job summary

HUD is building infrastructure to create RL training data and evals for frontier AI agents and a marketplace to sell them to labs. We seek a Research Manager to lead research engineers, shape data quality methods, and scale insights into repeatable workflows.

You will oversee problem definition, experiments, and conclusions, ensuring impact on agent training while staying close to technical work and cross-team collaboration.

Qualifications

• Experience leading technical research projects from question to working result.
• Experience mentoring researchers or research engineers while staying hands-on.
• Strong understanding of ML and RL and how data shapes model behavior.
• Experience with agent training data, evals, benchmarks, and validation infra.
• Sound experimental judgment to distinguish useful signals from misleading metrics.
• Excellent written communication to explain methods to researchers and partners.

Responsibilities

• Set research direction for data quality and how tasks, trajectories, rewards, and evals affect training.
• Lead researchers from problem definition through experiments and conclusions; coach for technical judgment.
• Design and review experiments linking model behavior to data, env, and reward design.
• Develop scalable methods to validate and improve training data, including audits and feedback loops.
• Partner with researchers, domain experts, and vendors to translate insights into workflows and standards.
• Communicate findings and tradeoffs clearly to help prioritize work across research areas.

Skills

Lead research projects

Mentor researchers

ML / RL expertise

Agent training data & evals

Experimental judgment

Strong written communication

Job description

HUD is building infrastructure to create RL training data and evals for frontier AI agents and a marketplace to sell them to labs. We seek a Research Manager to lead research engineers, shape data quality methods, and scale insights into repeatable workflows.

You will oversee problem definition, experiments, and conclusions, ensuring impact on agent training while staying close to technical work and cross-team collaboration.