Senior ML Evaluation & Data Platform Engineer (Remote)

Jobicy; United States; Remote; USD 180,000 - 240,000; Full time

Benefits offered by this job

Unlimited PTO

Equity

Health benefits

ESPP

401k

Offsite team retreats

Job summary

Hims & Hers is seeking a Senior Staff engineer to own end-to-end AI evaluation, data pipelines, and validation in healthcare AI products. You will champion statistically sound gates, regression testing, and robust data governance across ML teams.

You will lead cross‑disciplinary efforts, mentor engineers, and define standards for evaluation that scale to company-wide adoption. This role emphasizes trust, safety, and reproducibility in production AI.

Qualifications

• 10+ years of experience in ML infrastructure, data engineering, or evaluation/testing systems.
• Hands-on depth in evaluation systems: designing and calibrating LLM judges/scorers, building statistically sound regression-testing methodology.
• Hands-on depth in data pipeline engineering: dataset versioning, feature and benchmark pipelines, labeling and calibration workflows.
• Experience leading multi-team projects to completion, including navigating disagreements.
• Excellent communication and ability to persuade diverse audiences.
• Strong Python and statistical fluency to design robust testing frameworks for production decisions.

Responsibilities

• Own the evaluation as a whole, not just a slice of it.
• Set the technical direction for evaluation systems and the infrastructure that tracks failures.
• Design and scale data pipelines for ingestion, transformation, and labeling.
• Lead platform improvements and modernize core systems with broad impact.
• Mentor engineers and raise the technical bar across teams.
• Collaborate across ML engineering, data science, product, and legal groups.
• Develop reusable evaluation methodologies and publish learnings internally.

Skills

ML infrastructure

Data engineering

Evaluation systems

Python

Statistical fluency

Communication

Mentoring

Leadership

Tools

Databricks

MLflow

Unity Catalog

Job description

Hims & Hers is seeking a Senior Staff engineer to own end-to-end AI evaluation, data pipelines, and validation in healthcare AI products. You will champion statistically sound gates, regression testing, and robust data governance across ML teams.

You will lead cross‑disciplinary efforts, mentor engineers, and define standards for evaluation that scale to company-wide adoption. This role emphasizes trust, safety, and reproducibility in production AI.