Senior ML Systems Architect — Remote Evaluation & Trust

Hims, Inc.; Northern (KY); Hybrid; USD 150,000 - 210,000; Full time

Benefits offered by this job

Competitive salary

Unlimited PTO

Health benefits

ESPP

401k with match

Offsite team retreats

Job summary

Hims, Inc. is seeking a Senior Staff Engineer to own end-to-end AI evaluation, data pipelines, and governance across teams. You will define metrics, build robust judging/scoring frameworks, and scale datasets for trustworthy healthcare AI systems.

You will lead multi-team initiatives, mentor engineers, and drive automation to replace manual review with statistically sound gates. Strong Python and statistical fluency are required for production-grade decisions.

Qualifications

• 10+ years of experience in ML infrastructure, data engineering, or evaluation/testing systems, with impact across teams.
• Hands-on depth in evaluation systems: designing/calibrating LLM judges/scorers and regression-testing methodologies.
• Hands-on depth in data pipelines: dataset versioning, labeling/calibration workflows, and high-throughput ingestion/transformation.
• Experience leading multi-team projects and mentoring engineers.

Responsibilities

• Set the technical direction for evaluation systems, including metric/score design and regression methodology.
• Design and scale data pipelines and calibration workflows used by evaluation and data science teams.
• Lead cross-team initiatives to improve AI safety, governance, and production readiness.

Skills

ML infrastructure

Data engineering

Evaluation systems

Adversarial evaluation

Regression testing

Cross-team leadership

Mentoring

Python

Education

Bachelor's degree in CS or related field

Tools

Databricks

MLflow

Unity Catalog

Job description

Hims, Inc. is seeking a Senior Staff Engineer to own end-to-end AI evaluation, data pipelines, and governance across teams. You will define metrics, build robust judging/scoring frameworks, and scale datasets for trustworthy healthcare AI systems.

You will lead multi-team initiatives, mentor engineers, and drive automation to replace manual review with statistically sound gates. Strong Python and statistical fluency are required for production-grade decisions.