Senior ML Engineer: AI Evaluation & Reproducible Infra

Society for Human Resource Management (SHRM); Alexandria (VA); Hybrid; USD 100,000 - 130,000; Full time

Benefits offered by this job

Health benefits

Dental benefits

Vision benefits

Well-being programs

Health savings

Flexible spending

Retirement plan

Annual discretionary bonus

Job summary

SHRM is seeking a Senior Machine Learning Engineer, AI Evaluation to design and operate the measurement infrastructure for Applied AI Research. You will build scalable evaluation systems, manage model-versioning, and ensure reproducible benchmarks across multiple AI models in a collaborative, cross-functional environment.

You will partner with HR experts to translate professional standards into technically executable evaluation specifications and maintain auditable data infrastructure for

Qualifications

• Bachelor's degree in Computer Science, Data Science, Machine Learning, Engineering, or related field; relevant experience accepted.
• 7+ years of ML/LLM engineering, applied AI, or research infrastructure experience.
• Hands-on experience with multi-model LLM applications and evaluation frameworks.
• Strong knowledge of AI evaluation methodologies and reproducible systems.

Responsibilities

• Design, build, and maintain scalable engineering infrastructure for structured evaluations across AI/LLM families.
• Develop a unified orchestration layer for provider-agnostic evaluation across frontier models.
• Create robust evaluation and scoring frameworks with rubric-based scoring and model-as-judge approaches.
• Ensure reproducible experimentation with version pinning, logging, tracking, and drift detection.
• Collaborate with HR SMEs to translate standards into measurable evaluation specifications.

Skills

Python

ML/LLM

Data pipelines

Cloud platforms

Experimentation

Education

Bachelor's degree in CS/DS/Engineering

Master's degree preferred

Tools

BigQuery

Looker Looker Studio

Vertex AI

Git

Job description

SHRM is seeking a Senior Machine Learning Engineer, AI Evaluation to design and operate the measurement infrastructure for Applied AI Research. You will build scalable evaluation systems, manage model-versioning, and ensure reproducible benchmarks across multiple AI models in a collaborative, cross-functional environment.

You will partner with HR experts to translate professional standards into technically executable evaluation specifications and maintain auditable data infrastructure for