Responsible AI Evaluation Engineering Manager

Apple, Inc.; Cupertino (CA); On-site; USD 185,000 - 325,000; Full time

Benefits offered by this job

Medical and dental coverage

Retirement benefits

Employee stock purchase plan

Tuition reimbursement

Discounted products and services

Relocation assistance

Discretionary bonuses

Job summary

Apple, Inc. in Cupertino, CA, is seeking an Engineering Manager to lead a team responsible for evaluating the safety and inclusivity of Generative AI features.

You will guide ML engineers and data scientists, shaping evaluation criteria and data pipelines to deliver reliable insights for ship/no-ship decisions. You will collaborate across functions to operationalize policy, design experiments, and mentor a diverse team while upholding privacy, quality, and Apple's values as you drive responsible

Qualifications

• MS or PhD in Computer Science, Linguistics, Cognitive Science, HCI, Psychology, Mathematics, Physics, or a similar field with strong data collection and analysis; 4+ years of relevant work experience, or BA/BS with 8+ years and 2+ years of engineering-management experience
• Experience gathering and analyzing varied language, image, voice, video, and multimodal data, including human- and LLM-generated data
• Experience with LLM-Judges and human-in-the-loop annotation projects with guidelines and multiple labels
• Proficiency in data science, ML, analytics, Python & Pandas, and AI-assisted coding tools; familiarity with plotting/visualization libraries
• Strong cross-functional collaboration, prioritization, scheduling, and stakeholder communication
• Rigorous metric development, experimental interpretation, and executive-level presentation
• Comfort with handling sensitive or potentially offensive content

Responsibilities

• Develop metrics for evaluation of safety and fairness risks in generative models and Gen-AI features
• Design datasets, identify data needs, and expand data coverage with human and synthetic generation methods
• Develop sampling strategies and combine human annotation with auto-grading for high-quality insights
• Build data pipelines and collaborate cross-functionally for end-to-end safety evaluations
• Automate workflows, scale evaluation velocity, and support rapid product release cycles
• Distill findings into insights for product engineering and safety policy development
• Work with highly-sensitive content with exposure to offensive material

Job description

Apple, Inc. in Cupertino, CA, is seeking an Engineering Manager to lead a team responsible for evaluating the safety and inclusivity of Generative AI features.

You will guide ML engineers and data scientists, shaping evaluation criteria and data pipelines to deliver reliable insights for ship/no-ship decisions. You will collaborate across functions to operationalize policy, design experiments, and mentor a diverse team while upholding privacy, quality, and Apple's values as you drive responsible