Machine Learning Engineer, Evaluation

Apple; Cupertino (CA); On-site; USD 185,000 - 325,000; Full time

Benefits offered by this job

Med and dental coverage

Retirement benefits

Product discounts

Job summary

Apple seeks a talented professional to own and scale Responsible AI evaluation for global gen‑AI products. You will turn safety and inclusivity policy into measurable metrics and repeatable pipelines.

You will design evaluation datasets, combine human annotation with auto-grading, and build data pipelines for end-to-end safety assessments, collaborating across teams to accelerate safe releases. Strong data science and Python/Pandas skills are essential.

Qualifications

• MS or PhD in Computer Science, Linguistics, Cognitive Science, HCI, Psychology, Mathematics, Physics, or related field with strong data collection/analysis
• BA/BS with 8+ years relevant work experience, including 2+ years directly managing engineering teams
• Experience in responsible AI, safety/fairness in generative models, and diverse teams
• Experience gathering/analyzing varied language, image, voice, video, and multi-modal data
• Proficiency in data science, analytics, Python & Pandas, and AI-assisted coding tools

Responsibilities

• Develop evaluation metrics for safety and fairness risks in gen‑AI features
• Design datasets and scaling coverage with human and synthetic generation
• Create sampling strategies combining human annotation with auto-grading
• Implement data pipelines and collaborate cross-functionally for end-to-end safety evaluations
• Automate workflows to improve evaluation velocity and throughput
• Distill evaluation findings for product engineering and safety policy teams
• Work with sensitive content in a controlled, compliant manner

Skills

Responsible AI

Multimodal models

LLM-Judges

Human-in-the-loop

Data science

Python

Pandas

Cross-functional collaboration

Experiment design

Publications

Education

MS or PhD in Computer Science, Linguistics, Cognitive Science, HCI, Psychology, Mathematics, Physics (or related)

BA/BS with 8+ years relevant experience, including 2+ years managing eng teams

Tools

Python

Pandas

Job description

Own and scale Responsible AI evaluation for global generative AI products at Apple, turning safety and inclusivity policy into measurable metrics and repeatable pipelines.

Responsibilities

• Develop evaluation metrics for safety and fairness risks in generative models and Gen-AI features
• Design and evolve datasets, identifying data needs and scaling coverage using human and synthetic generation methods
• Create sampling strategies and combine human annotation with auto-grading to produce high-confidence insights at scale
• Use and implement data pipelines and collaborate cross-functionally to execute end-to-end safety evaluations
• Improve operational efficiency by automating manual workflows, optimizing evaluation velocity, and scaling evaluation infrastructure for rapid release cycles
• Distill evaluation findings into insights for product engineering teams and safety policy development
• Work with highly sensitive content, including exposure to offensive and controversial material

Requirements

• Experience working in the Responsible AI space
• Strong interest in building innovative, high-impact products and teams using interdisciplinary skills
• Curiosity about fairness and bias in generative AI, with motivation to improve equity
• Social Science or Humanities background, or demonstrable interest in socio-technical issues
• Prior scientific research and publication experience
• Proven record contributing to diverse teams in collaborative environments
• Experience with multimodal models, agentic systems, and/or diverse languages and locales
• Experience with compliance and regulatory needs in AI, specifically related to child safety, clinical/health applications, or companion chatbots
• Experience conducting large-scale benchmarking across models and capabilities
• MS or PhD (fields listed: Computer Science, Linguistics, Cognitive Science, HCI, Psychology, Mathematics, Physics, or a similar science or technology field) with a strong basis in scientific data collection and analysis, plus at least 4 years relevant work experience; or BA/BS with 8+ years relevant work experience, including 2+ years directly managing engineering teams
• Experience gathering and analyzing varied language, image, voice, video, and/or multi-modal data, including human- and LLM-generated data
• Experience with LLM-Judges and human-in-the-loop techniques, including designing human annotation projects, writing guidelines, and handling highly multi-labeled, nuanced, often conflicting data
• Proficiency in data science, machine learning, analytics, programming with Python & Pandas, and AI-assisted coding tools; experience with one or more plotting and visualization libraries
• Cross-functional collaboration skills to prioritize competing needs, set and maintain milestone schedules, and communicate with team members and internal/external stakeholders
• Rigorous approach to metric development, interpretation of experiments and evaluations, and presentation to executives
• Comfort leading a team working with sensitive or potentially offensive content

Technologies

• Python
• Pandas

Benefits

• Comprehensive medical and dental coverage
• Retirement benefits
• Range of discounted products and free services
• Reimbursement for certain educational expenses, including tuition
• Discretionary bonuses or commission payments and relocation (if eligible)
• Apple shareholder opportunity through participation in Apple’s discretionary employee stock programs
• Eligible for discretionary restricted stock unit awards
• Can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan

Pay & Benefits Notes

• Base pay range: $184,700 - $324,800 (yearly)
• Base pay determined within the range based on skills, qualifications, experience, and location
• Apple benefit, compensation, and employee stock programs are subject to eligibility requirements and other terms of applicable plans or programs
• Learn more about Apple Benefits