Senior ML Evaluation & Data Platform Architect

hims & hers; United States; On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Competitive salary & equity

Unlimited PTO

Health benefits

ESPP

401k

Offsite team retreats

Job summary

Hims & Hers is seeking a Senior Staff Engineer to own the end-to-end AI evaluation framework, including data ingestion, dataset versioning, and robust testing methodologies. You’ll set direction for judges, scorers, and statistical gates, partnering with clinical, product, and engineering teams to ship safe, validated AI services.

This leadership role emphasizes cross-team influence, scalable platform improvements, and mentoring engineers while upholding rigorous statistical standards and

Qualifications

• 10+ years in ML infrastructure, data engineering, or evaluation/testing systems.
• Hands-on depth in evaluation systems: designing/calibrating judges, scorers, regression testing.
• Hands-on depth in data pipeline engineering: dataset versioning, labeling/calibration workflows.
• Histor[y] of building standards that others adopt broadly.
• Experience leading multi-team projects to completion.
• Mentoring other engineers and raising technical bar across teams.
• Excellent communication and ability to tailor messages for different audiences.
• Strong Python and statistical fluency to design/defend testing frameworks.

Responsibilities

• Own the evaluation as a whole, not just a slice of it.
• Design and scale data pipelines: ingestion, transformation, labeling, calibration.
• Drive scalable, automated evaluation gates and regression tooling.
• Lead multi-team initiatives spanning engineering, product, and AI leadership.
• Define evaluation criteria for new AI services from scratch.
• Lead adversarial/red-team evaluation approaches to reduce risk.
• Turn vague cross-team pain points into shipped, reusable solutions.
• Grow the team and network; mentor engineers and share learnings.

Skills

10+ years experience

Evaluation systems

Data pipelines

Multi-team leadership

Excellent communication

Python

Statistics

Tools

Databricks

MLflow

Unity Catalog

Job description

Hims & Hers is seeking a Senior Staff Engineer to own the end-to-end AI evaluation framework, including data ingestion, dataset versioning, and robust testing methodologies. You’ll set direction for judges, scorers, and statistical gates, partnering with clinical, product, and engineering teams to ship safe, validated AI services.

This leadership role emphasizes cross-team influence, scalable platform improvements, and mentoring engineers while upholding rigorous statistical standards and