ML Evaluation Engineer, GenAI & Fairness Metrics

Apple Inc.; Austin, Northern (TX, KY); Hybrid; USD 140,000 - 190,000; Full time

Job summary

Apple Inc. seeks a Machine Learning Engineer focused on Evaluation for Wallet, Payments, and Commerce. You will define the evaluation framework, bias testing, and readiness gates ensuring high accuracy, robustness, and reliability for hundreds of millions of users across global markets.

The role emphasizes adversarial testing, audit-like sign-off processes, and collaboration with ML, Product, Privacy, and Legal teams to raise the quality bar for model deployment.

Qualifications

• MS in ML or related field preferred.
• 7+ years in ML evaluation or related work.
• 5+ years ML experience focused on evaluation.
• Experience designing evaluation frameworks beyond accuracy.
• Strong Python and experiment-tracking proficiency.

Responsibilities

• Define evaluation criteria and quality metrics for Wallet features.
• Design test sets covering diverse real-world scenarios and adversarial inputs.
• Develop robustness testing methodologies for distribution shift and drift.
• Own fairness evaluation end-to-end and gate launches on fairness criteria.
• Build benchmarks reflecting Wallet's global user base and spending patterns.
• Evaluate generative model outputs and establish sign-off criteria for launches.
• Synthesize results into actionable insights for product decisions.
• Collaborate with ML and Quality engineers to address failure modes early.

Skills

ML evaluation

Data analysis

Communication

Cross-functional collaboration

Education

MS in Machine Learning

PhD (preferred)

Tools

Python

MLflow

Weights & Biases

OCR pipelines

Job description

Apple Inc. seeks a Machine Learning Engineer focused on Evaluation for Wallet, Payments, and Commerce. You will define the evaluation framework, bias testing, and readiness gates ensuring high accuracy, robustness, and reliability for hundreds of millions of users across global markets.

The role emphasizes adversarial testing, audit-like sign-off processes, and collaboration with ML, Product, Privacy, and Legal teams to raise the quality bar for model deployment.