Algorithm Evaluation Manager

Apple Inc.; Sunnyvale (CA); On-site; USD 206,000 - 356,000; Full time

Benefits offered by this job

Stock options

Discretionary bonuses

Relocation assistance

Education reimbursement

Job summary

Apple Inc. in Sunnyvale, CA is seeking an Engineering Manager to lead a dynamic team focused on machine learning algorithms and large language model evaluation. You will guide data and evaluation pipelines to ensure models are accurate, robust, and performant.

The role emphasizes leadership, data curation, and strategic collaboration with cross-functional research and engineering teams to drive innovations in AI systems and deployment readiness.

Qualifications

• BS degree and minimum of 10 years relevant industry experience
• 2+ years of direct people management experience leading technical teams
• Proven experience in model evaluation, benchmarking, and A/B testing methodologies for ML models (CV or Foundation Models)
• Familiarity with design and architecture of ML inference pipelines and underlying infrastructure
• Hands-on experience designing and managing data curation strategies and human-in-the-loop annotation processes
• Strong analytical skills to identify trends, biases, and model improvement opportunities
• Excellent communication skills for technical and non-technical stakeholders

Responsibilities

• Lead, mentor, and grow a team of engineers and data specialists
• Define and execute evaluation strategy for CV and LM models
• Oversee end-to-end data lifecycle including data quality and annotation workflows
• Guide rigorous data analysis to troubleshoot model regressions
• Communicate progress and data needs to leadership and partner teams

Skills

Team leadership

Data curation

Model evaluation

A/B testing

Education

BS degree

Tools

Python

SQL

PyTorch

Job description

Sunnyvale, California, United States Machine Learning and AI

We are looking for an experienced and highly motivated Engineering Manager to lead a dynamic team focused on machine learning algorithms or Large Language Model (LM) evaluation. In this role, you will guide a team responsible for the critical data and evaluation pipelines that ensure our models are accurate, robust, and performant. The ideal candidate will bring a strong mix of technical leadership, expertise in data curation and annotation processes, and deep analytical skills. You will collaborate closely with cross-functional research and engineering teams, requiring exceptional communication and strategic thinking.

Description

As the Engineering Manager for this team, you will be at the forefront of our AI/ML development lifecycle. Your day-to-day responsibilities will include:

• Team Leadership: Lead, mentor, and grow a team of engineers and data specialists. Foster a culture of innovation, rigorous analysis, and continuous learning.
• Evaluation Strategy: Define and execute the evaluation strategy for both CV and LM models. Build robust, scalable evaluation pipelines that accurately reflect real-world performance.
• Data Pipeline Management: Oversee the end-to-end data lifecycle. This includes establishing data curation guidelines, managing data quality, and optimizing large-scale annotation workflows with external vendors or internal teams.
• Analytical Deep Dives: Guide the team in performing rigorous data analysis to troubleshoot model regressions, uncover data quality issues, and identify opportunities for algorithmic improvements.
• Strategic Alignment: Act as the primary point of contact for your team, communicating progress, bottlenecks, and strategic data needs to leadership and partner teams.

Minimum Qualifications

• Education & Experience: BS and a minimum of 10 years relevant industry experience
• Management Experience: 2+ years of direct people management experience, with a track record of hiring, mentoring, and leading high-performing technical teams.
• Domain Expertise: Proven experience in model evaluation, benchmarking, and A/B testing methodologies for machine learning models (Computer Vision or Foundation Models).
• Inference Infrastructure: Familiarity with the design and architecture of machine learning inference pipelines and underlying infrastructure.
• Data & Annotation: Hands-on experience designing and managing data curation strategies and human-in-the-loop annotation processes.
• Data Analysis: Strong analytical skills with the ability to dive deep into datasets to identify trends, biases, and areas for model improvement.
• Communication: Excellent verbal and written communication skills, with the ability to translate complex technical concepts to both technical and non-technical stakeholders.

Preferred Qualifications

• Advanced Degree: PhD in Computer Science, Machine Learning, or a related field.
• Deep Domain Knowledge: Expertise in both Computer Vision (CV) algorithms and Large Language Model (LM) evaluation methodologies (e.g., RLHF, prompt evaluation).
• Scale & Operations: Experience scaling large data operations, managing complex annotation workflows, and working directly with external data vendors.
• Technical Stack: Familiarity with Python, SQL, and ML frameworks (e.g., PyTorch) to effectively review technical work and guide engineering decisions.
• Cross-Functional Leadership: Demonstrated ability to drive strategic alignment across downstream product teams, ML researchers, and platform engineers in a highly matrixed environment.

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $206,200 and $356,400, and your base pay will depend on your skills, qualifications, experience, and location.
Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan. You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits
Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant

At Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.
Learn about accessibility in Apple’s workplace
Learn about reasonable accommodations for job applicants

Apple accepts applications to this posting on an ongoing basis.