Machine Learning Engineer, Evaluation - Responsible AI and Safety

Apple; Cambridge (MA); On-site; USD 166,000 - 292,000; Full time

Benefits offered by this job

Stock programs

Medical & dental coverage

Retirement benefits

Employee discounts

Education reimbursement

Relocation assistance

Job summary

Apple in Cambridge, MA is seeking a Senior Machine Learning Engineer to lead Responsible AI evaluation for global generative AI products. You will design metrics, datasets and data pipelines to support ship/no-ship decisions.

You will run hands-on evaluation, analyze results, and translate findings into actionable insights for product engineering and safety policy, working with sensitive content to strengthen user trust and safety.

Qualifications

• Experience with evaluating safety and fairness in generative AI.
• Ability to design and run evaluation studies across models and features.
• Ability to translate complex findings into actionable product and policy insights.

Responsibilities

• Design evaluation approaches quantifying safety and inclusivity for generative AI features.
• Use human and automated signals to iterate on evaluation outcomes.
• Translate results into actionable insights for product engineering and safety policy.

Skills

Multimodal models

Agentic systems

Regulatory compliance in AI

Large-scale benchmarking

Job description

Apple is seeking a Senior Machine Learning Engineer to lead Responsible AI evaluation for global generative AI products. The role focuses on assessing, measuring, and improving safety and inclusivity, with an emphasis on building evaluation metrics, datasets, methods, and data pipelines that support product launch decisions.

What You’ll Do

• Operationalize policy and risk assessments into measurable criteria, including designing datasets, developing LLM and human-in-the-loop analysis methods, and implementing robust data pipelines that produce reliable, scalable insights for ship/no-ship decisions.
• Run hands-on evaluation work before and after launch by analyzing results, investigating failures, and using both human and automated feedback to improve generative feature safety and strengthen user trust.
• Maintain technical excellence, enable leadership with data-driven safety recommendations, and work through complex engineering roadblocks.
• Defend safety metrics with stakeholders and decision-makers.

Core Responsibilities in Evaluation Execution

• Design evaluation approaches that quantify safety and inclusivity for generative AI features.
• Use human and automated signals to iterate on evaluation outcomes and decision-making signals.

Requirements

• Develop metrics to evaluate safety and fairness risks in generative models and Gen-AI features.
• Design datasets, determine data needs, and use creative approaches to scale data coverage through human and synthetic generation.
• Define sampling strategies and pair human annotation with auto-grading to produce high-quality, high-confidence insights at scale.
• Design, implement, and optimize data pipelines, collaborating cross-functionally to deliver end-to-end safety evaluations.
• Improve operational excellence by automating manual workflows and increasing evaluation velocity to support rapid product release cycles.
• Translate complex findings into actionable insights for product engineering teams and safety policy development.
• Work with highly sensitive content, including exposure to offensive and controversial material.

Preferred Qualifications

• Experience with multimodal models, agentic systems, and/or diverse languages and locales.
• Experience with compliance and regulatory needs in AI, particularly child safety, clinical/health applications, or companionship chatbots.
• Experience conducting large-scale benchmarking across models and capabilities.

Compensation & Benefits

• Base pay range: USD $165,800 to $292,400 per year, depending on skills, qualifications, experience, and location.
• Opportunity to become an Apple shareholder through Apple’s discretionary employee stock programs, including discretionary restricted stock unit awards, and the ability to purchase Apple stock at a discount via the Employee Stock Purchase Plan.
• Comprehensive medical and dental coverage.
• Retirement benefits.
• A range of discounted products and free services.
• Reimbursement for certain educational expenses, including tuition, for formal education related to advancing your career.
• Eligibility for discretionary bonuses or commission payments and relocation may apply.

Additional Information

Apple benefit, compensation, and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program. Learn more about Apple Benefits.

Location & Work Mode

Cambridge, MA (onsite).