Senior ML Evaluation & Data Platform Architect

hims-and-hers; United States; Remote; USD 180,000 - 280,000; Full time

Benefits offered by this job

Competitive salary & equity

Unlimited PTO

Comprehensive health benefits

ESPP

401k with employer matching

Offsite team retreats

Job summary

Hims & Hers is seeking a Senior Staff engineer to own end-to-end evaluation for AI models in a regulated healthcare environment. You will lead data pipelines, evaluation infrastructure, and red-team testing to ensure models are effective and safe for patients.

This leadership role requires cross-functional collaboration, technical direction, and mentorship across ML, data science, and product teams. The role emphasizes scalable data ingestion, rigorous statistical methods, and automated gates

Qualifications

• 10+ years of experience in ML infrastructure, data engineering, or evaluation/testing systems.
• Hands-on depth in evaluation systems: designing and calibrating LLM judges/scorers, and regression-testing methodology.
• Hands-on depth in data pipeline engineering: dataset versioning, feature/benchmark pipelines, labeling/calibration workflows, and high-throughput ingestion.
• A history of building standards and guiding multi-team projects beyond a single scope.
• Experience leading multi-team projects to completion, including resolving technical disagreements and mentoring engineers.
• Excellent communication—able to tailor messages for diverse audiences and gain buy-in early.
• Strong Python and statistical fluency to support production-grade testing frameworks.

Responsibilities

• Own the evaluation as a whole, not just a slice of it.
• Design and scale data pipelines for ingestion, transformation, dataset versioning, labeling, and calibration.
• Drive scalable, automated evaluation gates and replacement of manual review processes.
• Lead cross-team initiatives for new AI services and risk-reduction through adversarial testing.
• Mentor engineers and raise the technical bar across ML, data science, and product partners.
• Communicate findings and standards across teams and document methodologies for reuse.

Skills

ML infrastructure

Data engineering

Evaluation systems

Python

Statistical fluency

Mentoring engineers

Leadership

Communication

Tools

Databricks

MLflow

Unity Catalog

Job description

Hims & Hers is seeking a Senior Staff engineer to own end-to-end evaluation for AI models in a regulated healthcare environment. You will lead data pipelines, evaluation infrastructure, and red-team testing to ensure models are effective and safe for patients.

This leadership role requires cross-functional collaboration, technical direction, and mentorship across ML, data science, and product teams. The role emphasizes scalable data ingestion, rigorous statistical methods, and automated gates