Senior Manager, LLM Evaluation & Quality Services

Apple Inc.; Cupertino (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

Apple Inc. in Cupertino, California seeks a hands-on Senior Manager, Evaluation, to build and lead a team of applied scientists and ML engineers responsible for measuring quality of a new multi-year initiative.

You will own evaluation methods, datasets, and automated graders, design LLM-based rubrics, and collaborate with engineering, QA, and PM teams to drive reliable release decisions. This leadership role emphasizes mentoring, recruitment, and shaping what 'good' and 'shippable' mean as

Qualifications

• Build automated grading systems that scale to large, frequently changing datasets.
• Define metrics and evaluation sets that reflect real user behavior and predict real-world quality.
• Turn evaluation findings into action by partnering with engineering and ML teams to diagnose failures and ship fixes.
• Contribute evaluation results to release processes as explicit quality gates.
• Grow senior individual contributors and technical leads from within a team, and recruiting strong talent in competitive hiring markets.

Responsibilities

• Lead a team designing and running evaluation for a conversational experience.
• Own evaluation methods for the layers in your area, including LLM-based autograders and rubrics.
• Set technical direction, build roadmaps, and translate strategy into concrete deliverables.
• Collaborate with engineering, applied ML, QA, and PM teams to present quality status and trade-offs to senior leadership.

Skills

People management

LLM evaluation

Experiment design

A/B testing

Statistical analysis

Communication

Applied ML

Leadership

Education

MS/PhD in CS / ML / Stats

Job description

Apple Inc. in Cupertino, California seeks a hands-on Senior Manager, Evaluation, to build and lead a team of applied scientists and ML engineers responsible for measuring quality of a new multi-year initiative.

You will own evaluation methods, datasets, and automated graders, design LLM-based rubrics, and collaborate with engineering, QA, and PM teams to drive reliable release decisions. This leadership role emphasizes mentoring, recruitment, and shaping what 'good' and 'shippable' mean as