Director of AI Evaluation & Quality Assurance

Apple Inc.; Cupertino (CA); On-site; USD 260,000 - 380,000; Full time

Job summary

Apple Inc. seeks a Director to build and lead the evaluation and quality assurance function for a major multi-year engineering initiative.

This first-key-hire role will define end-to-end quality, build automation and evaluation infrastructure, and scale an organization with minimal scaffolding. You will partner with engineering, applied ML, and program management to set standards for what is considered done and shippable, balancing strategic risk with ambitious delivery across the product

Qualifications

• 12+ years of experience in data science, ML evaluation, QA, or test engineering in a technical product environment.
• Proven experience building evaluation or QA functions from the ground up, including metrics, tooling, and org structure.
• Deep experience with LLM-based autograders, rubric design, evaluation set development, validation, calibration, and alignment.
• Extensive experience with traditional human evaluation methods, logging, and AB testing at scale.
• Experience evaluating conversational AI, ranking, or search systems; ideal if agentic/tool-use systems are included.
• BS in CS/Engineering or equivalent; experience with third-party API integrations QA.
• Excellent written and verbal communication with senior leadership.

Responsibilities

• Design metrics and autograders that scale to large, dynamic datasets.
• Wiring evaluation results into release processes as explicit quality gates.
• Catching and preventing quality regressions across the product surface.
• Partner with engineering and applied ML to define quality and done.
• Navigate ambiguity and build structure for a new function space.
• Make resource allocation calls to maximize risk reduction.

Skills

QA leadership

LLM evaluation

A/B testing

Data science

Machine learning evaluation

Requirements gathering

Education

BS in Computer Science or related field

Tools

Autograders design

Rubrics design

Logging infrastructure

Job description

Apple Inc. seeks a Director to build and lead the evaluation and quality assurance function for a major multi-year engineering initiative.

This first-key-hire role will define end-to-end quality, build automation and evaluation infrastructure, and scale an organization with minimal scaffolding. You will partner with engineering, applied ML, and program management to set standards for what is considered done and shippable, balancing strategic risk with ambitious delivery across the product