AIML Evaluation Engineer - Shape Apple Intelligence

Apple Inc.; Cupertino (CA); On-site; USD 150,400 - 277,600; Full time

Benefits offered by this job

Medical and dental coverage

Retirement benefits

Employee stock programs

Job summary

Apple is seeking a Software Engineer in AIML Evaluation to improve evaluation systems for Apple Intelligence. You will collaborate with product managers, backend engineers, data scientists and other stakeholders to enhance product quality and user experience across Apple platforms.

You will design benchmarks, develop scalable evaluation tooling, and run experiments to guide model development priorities, aligning with product goals and quality standards.

Qualifications

• 3+ years of professional software engineering experience building and operating production systems.
• Ability to tackle complex challenges and deliver innovative solutions.
• Fluent use of AI-assisted development tools to work effectively at scale.
• Experience designing and scaling distributed systems (load balancing, active-active topologies).
• Experience with RESTful API design, SQL and NoSQL databases.
• BS or MS in Computer Science, Computer Engineering, or equivalent.

Responsibilities

• Benchmark Design & Development: Design and implement evaluation benchmarks, metrics, and test suites for model capabilities.
• Product‑Aligned Evaluation: Develop evaluation methods that reflect real product settings and predict user quality.
• Evaluation Methodology Research & Tooling: Research and apply state‑of‑the‑art evaluation techniques and build reusable tools.
• Experimental Analysis: Design and execute experiments comparing model capabilities and guide development priorities.
• Cross‑Team Collaboration: Work with model training, data, and product teams to inform training and product quality.

Skills

AI/ML evaluation

RESTful API design

SQL/NoSQL databases

Distributed systems

Critical thinking

Code review tools

Education

BS/MS in Computer Science or Computer Engineering

Tools

None

Job description

Apple is seeking a Software Engineer in AIML Evaluation to improve evaluation systems for Apple Intelligence. You will collaborate with product managers, backend engineers, data scientists and other stakeholders to enhance product quality and user experience across Apple platforms.

You will design benchmarks, develop scalable evaluation tooling, and run experiments to guide model development priorities, aligning with product goals and quality standards.