Senior ML Evaluation & Data Pipelines Lead

Hims & Hers; United States; Remote; USD 180,000 - 240,000; Full time

Benefits offered by this job

Unlimited PTO

Healthcare Benefits

401(k) match

Stock options

Flexible/remote work

Job summary

Hims & Hers seeks a Senior Staff Engineer to own end-to-end AI/ML evaluation infrastructure in a regulated healthcare context. You will shape data pipelines, evaluation metrics, judges, and the systems that ensure models are safe to ship to patients.

You will lead cross‑team initiatives, mentor engineers, and define scalable, auditable approaches, collaborating with ML, data science, product, and legal to raise the technical bar.

Qualifications

• 10+ years of experience in ML infrastructure, data engineering, or evaluation/testing systems.
• Hands-on depth in evaluation systems: designing/calibrating LLM judges/scorers, regression-testing, human-label agreement, adversarial evaluation.
• Hands-on depth in data pipeline engineering: dataset versioning, feature/benchmark pipelines, labeling/calibration workflows.

Responsibilities

• Own the evaluation end-to-end, including metrics, judges, scorers, and regression methodology.
• Design and scale data pipelines: ingestion, transformation, dataset versioning, labeling and calibration workflows.
• Lead multi‑team initiatives and cross‑functional alignment on evaluation strategy and data engineering practices.
• Mentor other engineers and raise the technical and statistical bar across teams.

Skills

ML infrastructure

Data engineering

Evaluation/testing systems

Python

Leadership

Mentoring

Communication

Regulated healthcare familiarity

Tools

Databricks

MLflow

Unity Catalog

Job description

Hims & Hers seeks a Senior Staff Engineer to own end-to-end AI/ML evaluation infrastructure in a regulated healthcare context. You will shape data pipelines, evaluation metrics, judges, and the systems that ensure models are safe to ship to patients.

You will lead cross‑team initiatives, mentor engineers, and define scalable, auditable approaches, collaborating with ML, data science, product, and legal to raise the technical bar.