AIML - Sr Machine Learning Platform Engineer, Evaluation

Apple, Inc.

Seattle (WA)

On-site

USD 180,000 - 280,000

Full time

10 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Apple, Inc. is seeking a highly skilled machine learning engineer to join AIML Evaluation.

You will build systems that evaluate and refine foundation models and agents, design benchmarks, simulators, and prompt optimization pipelines that drive quality across Apple's AI experiences, and collaborate across product and foundation model teams to improve data, environments, and reward signals. In this role, you will help connect failure analysis to measurable model improvements, implementing offline

Qualifications

  • 6+ years of professional software engineering experience, preferably in ML or related field.
  • Experience solving ambiguous problems end-to-end: define problem and metric, build solution, demonstrate impact.
  • Experience leading technical work across multiple codebases, teams, and organizations.
  • Experience building and maintaining ML infrastructure for evaluation, training, or deployment.

Responsibilities

  • Build systems that connect failure analysis to measurable model and agent improvements.
  • Develop benchmarks, evaluators, simulation environments, and prompt/context optimization pipelines.
  • Collaborate with product teams and foundation model team to close the loop between observation and improvement.

Skills

Machine learning
Distributed systems
Problem solving
Leadership
ML infrastructure

Education

Bachelor's or Master's in Computer Science

Tools

Python
PyTorch
JAX

Job description

We are seeking a highly skilled and experienced machine learning engineer to join AIML Evaluation to build the systems that evaluate and refine Apple's foundation models and agents.
As a key member of the team, you will help design and develop benchmarks, evaluators, simulation environments, and prompt and context optimization pipelines that drive quality improvements across Apple's AI experiences.
You will collaborate with product teams and the foundation model team to close the loop between observation and improvement, contributing datasets, environments, and reward signals that drive model and agent quality.

Most model evaluation stops at a scorecard. The failing traces get thrown away, even though they're the most relevant training signal we have. Our team puts them to work. We evaluate across offline benchmarks, device-in-the-loop simulation, and on-device observation in production, and we turn what we find into persona-grounded datasets and reward signals for foundation model post-training.

In this role, you will build the systems that connect failure analysis to measurable model and agent improvements across a broad range of AI products at Apple. The path from finding a problem to fixing it is short: the loop you build finds the failure, produces the data and checks the result. The work reaches millions of users.

Minimum Qualifications
  • Strong background in machine learning and distributed systems.
  • A proven track record of solving ambiguous problems end to end: defining the problem and the metric, building the solution, and showing measured impact.
  • Experience leading technical work across multiple codebases, teams, and organizations.
  • Experience building and maintaining ML infrastructure for evaluation, training, or deployment.
  • 6+ years of professional experience as a software engineer, preferably in machine learning or a related field. Bachelor's or Master's degree in Computer Science or a related field, or equivalent practical experience.
Preferred Qualifications
  • Experience with LLM evaluation, LLM-as-judge, or reward modeling.
  • Experience with prompt optimization, agent harness development, or post-training (SFT, DPO, RLHF).
  • Proficiency in Python and ML frameworks such as PyTorch or JAX.
  • Experience with agentic systems, simulation environments, or trajectory-based data generation.
  • Familiarity with on-device or privacy-preserving ML.
  • Understanding of how training data is used in post-training and how data choices affect model behavior.
  • Excellent communication skills.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior ML Evaluation Platform Engineer
Senior ML Evaluation Platform Engineer

Apple, Inc. • Seattle (WA)

On-site
USD 180,000 - 280,000
Machine Learning Evaluation Engineer
Machine Learning Evaluation Engineer

Apple Inc. • Sunnyvale (CA)

On-site
USD 150,000 - 278,000
Medical & Dental coverage
Employee stock programs
Relocation support
+1
AIML - Sr Software Engineer - AI, Evaluation
AIML - Sr Software Engineer - AI, Evaluation

Apple, Inc. • Cupertino (CA)

On-site
USD 150,000 - 278,000
Comprehensive medical and dental
Retirement benefits
Employee stock programs
+2
Machine Learning Engineer, Human Centered AI - Evaluations & Insights
Machine Learning Engineer, Human Centered AI - Evaluations & Insights

Apple Inc. • Seattle (WA)

On-site
USD 142,000 - 263,000
AIML - Data Scientist, Evaluation
AIML - Data Scientist, Evaluation

Apple, Inc. • Cupertino (CA)

On-site
USD 147,400 - 272,100
Employee stock programs
Comprehensive medical coverage
Retirement benefits
+1
AIML Researcher/Engineer - Foundation Model Post-Training
AIML Researcher/Engineer - Foundation Model Post-Training

Apple Inc. • Seattle (WA)

On-site
USD 175,000 - 309,000
Medical and dental coverage
Retirement benefits
Employee stock plan
AIML Researcher/Engineer - Foundation Model Post-Training
AIML Researcher/Engineer - Foundation Model Post-Training

Apple Inc. • Cupertino (CA)

On-site
USD 185,000 - 325,000
Stock programs
Medical coverage
Tuition reimbursement
+2
Sr. Applied Scientist, AI Evaluation & Quality Systems
Sr. Applied Scientist, AI Evaluation & Quality Systems

Apple Inc. • Seattle (WA)

On-site
USD 142,000 - 263,000
Apple Benefits
Relocation assistance
Employee stock purchase plan
+2
AIML - Data Scientist, Evaluation
AIML - Data Scientist, Evaluation

Apple, Inc. • Cupertino (CA)

On-site
USD 150,000 - 278,000
Stock programs
Medical and dental coverage
Retirement benefits
+4
Sr Engineering Program Manager, Apple Intelligence Evaluations
Sr Engineering Program Manager, Apple Intelligence Evaluations

Apple Inc. • Cupertino (CA)

On-site
USD 145,000 - 264,000