Senior AI Evaluation & Benchmarking Lead

Apple; Cambridge; On-site; GBP 100,000 - 140,000; Full time

Job summary

Apple seeks a Senior Manager to lead evaluation development and data science for Siri. You will shape measurement systems across multiple Apple platforms, guiding teams, and collaborating with Finance, Annotation Ops, and international partners to ensure high-quality human-in-the-loop evaluations.

Join a cross-disciplinary team focused on robust metrics, privacy-first design, and scalable tooling to manage rich data assets while delivering state-of-the-art evaluation capabilities for users

Qualifications

• Minimum Qualifications: Agentic coding proficiency to achieve data-science, data collection and visualization tasks.
• Good understanding of metrics, crowd science, data collection, annotation analysis, statistics.
• Ability to work independently and cross-functionally to integrate in partner team reporting systems and pipelines.
• Excellent communication skills and the ability to thrive in a highly collaborative work environment.

Responsibilities

• Design and execute efficient data collection processes using humans in the loop.
• Lead annotation efforts for various languages and devices.
• Plan and manage the budget of annotator resourcing with Finance, Annotation Ops, and International team partners.
• Track and improve the quality of the human judgments through revision of annotator training materials, clear annotation questions, annotator training, and efficient reviewing mechanisms.
• Collaborate with other engineering teams to design and build a tooling ecosystem for managing and browsing rich datasets.

Job description

Apple seeks a Senior Manager to lead evaluation development and data science for Siri. You will shape measurement systems across multiple Apple platforms, guiding teams, and collaborating with Finance, Annotation Ops, and international partners to ensure high-quality human-in-the-loop evaluations.

Join a cross-disciplinary team focused on robust metrics, privacy-first design, and scalable tooling to manage rich data assets while delivering state-of-the-art evaluation capabilities for users