Gen AI Quality Scientist: Evaluation & Metrics

Apple Inc.; Sunnyvale (CA); On-site; USD 185,000 - 325,000; Full time

Benefits offered by this job

Stock programs

RSU awards

Employee stock purchase plan

Medical and dental coverage

Retirement benefits

Product discounts

Educational expense reimbursement

Relocation assistance

Discretionary bonuses

Job summary

Apple is hiring for a role focused on evaluating the quality of generative imaging features. You will design evaluation sets, run human annotation, and develop automated metrics including LLM-as-judge systems, translating findings into decision-ready signals for model development.

Responsibilities include owning the evaluation lifecycle end to end, maintaining golden sets, building labeling tools, and calibrating automated outputs against human ratings.

Qualifications

• MS degree with at least 5 years of industry experience.
• Experience designing and analyzing ML evaluations and experiments.
• Strong Python and SQL skills; ability to build evaluation pipelines.

Responsibilities

• Own the quality evaluation lifecycle end to end, from design to recommendations.
• Establish measurement foundation with golden and evaluation sets.
• Run human evaluation with guidelines and labeling tools; monitor annotator quality.
• Automate metrics and LLM-as-judge systems; calibrate against human labels.
• Produce decision-ready results with model-vs-model comparisons and analyses.
• Collaborate across ML, engineering, and design to define quality standards.

Skills

Python

SQL

Data science

ML evaluation

Statistics

Communication

Cross-functional collaboration

Adaptability

Education

MS

Tools

LLM-as-judge systems

LLMs

VLM

3D Gaussian splats

diffusion models

Job description

Apple is hiring for a role focused on evaluating the quality of generative imaging features. You will design evaluation sets, run human annotation, and develop automated metrics including LLM-as-judge systems, translating findings into decision-ready signals for model development.

Responsibilities include owning the evaluation lifecycle end to end, maintaining golden sets, building labeling tools, and calibrating automated outputs against human ratings.