Software Engineer, Evaluation

Apple Inc.; Cupertino (CA); On-site; USD 150,000 - 278,000; Full time

Benefits offered by this job

Comprehensive medical and dental

Employee stock programs

Education reimbursement

Job summary

Apple Inc. is seeking a Software Engineer in AIML Evaluation in Cupertino, CA. You will design evaluation systems to improve Apple Intelligence, build benchmarks, and scale tooling for model assessment across products.

Collaborate with product managers, data scientists, and engineers to ensure high-quality AI experiences. The role requires 3+ years of software engineering, expertise in distributed systems and RESTful APIs, and fluency with AI-assisted development tools.

Qualifications

• 3+ years of professional software engineering experience building and operating production systems.
• Ability to tackle complex challenges, think critically, and deliver innovative solutions.
• Fluent use of AI-assisted development tools to work effectively at scale.
• Experience designing and scaling distributed systems (load balancing, active-active topologies).
• Experience with RESTful API design, SQL and NoSQL databases.
• BS, MS in Computer Science, Computer Engineering, or equivalent degree with applicable experience.

Responsibilities

• Benchmark Design & Development: Design and implement evaluation benchmarks, metrics, and test suites that rigorously measure model capabilities across reasoning, knowledge, code, and agentic workflows.
• Product-Aligned Evaluation: Develop evaluation methods that capture how models behave in real product settings, and validate that evaluation metrics predict user-perceived quality and product outcomes.
• Evaluation Methodology Research & Tooling: Research and apply state-of-the-art evaluation techniques — including scoring frameworks, model-based judging, and contamination-resistant benchmark design. Build reusable tools, scorer libraries, and analysis frameworks that scale across the team’s benchmark portfolio.
• Experimental Analysis: Design and execute rigorous experiments comparing model capabilities, engage with third-party vendors on benchmarking, and perform detailed gap analysis to guide model development priorities.
• Cross-Team Collaboration: Work closely with model training, training data, and product teams to ensure evaluation insights inform training strategies, data decisions, and product quality improvements.

Skills

Software engineering experience

Distributed systems design

RESTful API design

SQL and NoSQL databases

AI-assisted development tools

Education

BS/MS in CS/CE or equivalent

Tools

Coding agents

Code review assistants

Job description

Cupertino, California, United States Software and Services

At Apple, great ideas have a way of becoming great products, services, and customer experiences very quickly. Bring passion and dedication to your job and there’s no telling what you could accomplish. Do you want to make Siri and Apple Intelligence features smarter for our users? Contribute to the next revolution in AI and Machine Learning. We're a dynamic team seeking engineers who are enthusiastic about influencing the future of Apple Intelligence.

Description

As a Software Engineer in AIML Evaluation, you will:- Work closely with product managers, backend engineers, data scientists and other stakeholders to improve the overall quality and user experience of numerous Apple products.- Design and develop evaluation system and framework to enhance the quality of Apple Intelligence, ensuring our AI/ML models meet the highest standards.- Tuning evaluation infrastructure, tightening operational controls, improving auditability and debug trails, and scaling the workflows .- You’ll play a pivotal role in developing Apple Intelligence, driving the next generation of groundbreaking products across all Apple platforms.

Responsibilities

• Benchmark Design & Development: Design and implement evaluation benchmarks, metrics, and test suites that rigorously measure model capabilities across reasoning, knowledge, code, and agentic workflows.
• Product-Aligned Evaluation: Develop evaluation methods that capture how models behave in real product settings, and validate that evaluation metrics predict user-perceived quality and product outcomes.
• Evaluation Methodology Research & Tooling: Research and apply state-of-the-art evaluation techniques — including scoring frameworks, model-based judging, and contamination-resistant benchmark design. Build reusable tools, scorer libraries, and analysis frameworks that scale across the team’s benchmark portfolio.
• Experimental Analysis: Design and execute rigorous experiments comparing model capabilities, engage with third-party vendors on benchmarking, and perform detailed gap analysis to guide model development priorities.
• Cross-Team Collaboration: Work closely with model training, training data, and product teams to ensure evaluation insights inform training strategies, data decisions, and product quality improvements.

Minimum Qualifications

• 3+ years of professional software engineering experience building and operating production systems
• Ability to tackle complex challenges, think critically, and deliver innovative solutions.
• Fluent use of AI-assisted development tools (coding agents, code review assistants, etc.) to work effectively at scale
• Experience designing and scaling distributed systems (load balancing, active-active topologies, capacity planning, throughput-bound services)
• Experience with RESTful API design, SQL and NoSQL databases
• BS, MS in Computer Science, Computer Engineering, or equivalent degree with applicable experience

Preferred Qualifications

• Eager to explore and apply cutting-edge tech solutions.
• Experience working on large, cross-functional, and multi-team projects.
• Proven ability to operate independently on ambiguous, open-ended problems where the right answer is not obvious.
• Striving to elevate every aspect of the user experience.

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $150,400 and $277,600, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan. You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant

At Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.

Learn about accessibility in Apple’s workplace

Learn about reasonable accommodations for job applicants