Machine Learning Engineer, Evaluation and Quality, Creativity Apps

Apple, Inc.; California; On-site; USD 142,000 - 263,000; Full time

Benefits offered by this job

Medical and dental coverage

Employee stock programs

Relocation assistance

Job summary

Apple, Inc. in San Diego, CA seeks a Machine Learning Engineer focused on evaluating and ensuring the quality of ML models and AI features for Creativity Apps. You will define metrics, curate datasets, and build scalable evaluation infrastructure, collaborating across engineering, product, and design teams.

You will drive end-to-end quality from model development to customer shipping, communicating findings to technical and non-technical stakeholders while staying current with ML/AI advancements.

Qualifications

• 5+ years of ML or software engineering experience focused on model evaluation.
• Experience with evaluation methodology for ML models, LLMs, and AI products or features.
• Strong understanding of the ML development cycle, from data and training to evaluation, deployment, and monitoring.
• Fundamentals in ML and LLMs, including agentic workflows.
• Experience in data curation and dataset quality improvements.
• Experience designing evaluation metrics.
• Experience building evaluation infrastructure at scale.
• Fluency in quality reporting to technical and non-technical audiences.
• Strong communication and cross-functional collaboration.
• Proficient in Python and/or Swift.
• Bachelor's degree in CS/DS/ML or related field, or equivalent experience.

Responsibilities

• Define evaluation strategy and metrics for ML models, LLMs, and AI features.
• Curate and create evaluation datasets, measure and improve dataset quality.
• Design, build, maintain, and optimize evaluation infrastructure at scale.
• Apply ML/LLMs and agentic workflows to evaluation.
• Drive end-to-end evaluation and quality from model checkpoints to shipping features.
• Report quality results with ship/no-ship recommendations.
• Partner with cross-functional teams and navigate ambiguity.

Skills

ML model evaluation

Python

Swift

Cross-functional collab

Data curation

LLMs and AI

Agentic workflows

Evaluation metrics

ML development cycle

Quality reporting

Education

Bachelor's degree in CS/DS/ML or related field

Tools

Python

Swift

Job description

Machine Learning Engineer, Evaluation and Quality, Creativity Apps

San Diego Metro Area, California, United States Software and Services

Imagine what you could do here. At Apple, new ideas have a way of becoming outstanding products, services, and customer experiences very quickly. Bring passion and dedication to your job and there's no telling what you could accomplish.The Creativity Apps team is looking for a Machine Learning Engineer focused on the quality and evaluation of ML models and AI features in our creativity applications. You will define how we measure whether the product and AI features are good enough to ship: designing the evaluation strategy and metrics, building the datasets and infrastructure behind them, and driving end-to-end quality from model development through to the feature in customers' hands. You will have the unique and rewarding opportunity to help shape upcoming products that will delight and inspire millions of creators every day.

Description

In this role, you will own the evaluation and quality of AI features across Creativity Apps. You will work closely with Machine Learning engineering, infrastructure, product, and design teams to understand each model and use case, and translate them into evaluation methodology, testing strategy, and quality metrics that reflect what creators actually experience.You will:

• Define evaluation strategy and metrics for ML models, LLMs, and AI features, tailored to the model, the product, and its use cases.
• Curate and create evaluation datasets, measure and improve dataset quality, and run iterative improvement loops so that datasets stay representative of each product.
• Design, build, maintain, and optimize evaluation infrastructure at scale, with reproducible runs, tracked baselines, and comparisons between model versions.
• Apply ML, LLMs, and agentic workflows to the evaluation itself, including automated judging, test generation, and analysis, to make evaluation more efficient, more accurate, and more scalable.
• Drive end-to-end evaluation and quality of AI features, from model checkpoints to integrated, shipping features, including triaging issues and driving them to resolution.
• Report quality clearly, turning evaluation results into ship/no-ship risk and recommendations for both technical and non-technical audiences.
• Partner with and drive cross-functional teams, comfortable with ambiguity and making educated, critical judgments.

Your curiosity about the latest developments in ML and generative AI, and your drive to apply new technology to practical problems, will set you apart.

Minimum Qualifications

• 5+ years of industry experience in machine learning or software engineering, with a focus on ML/AI model evaluation and quality.
• Experience with evaluation methodology for ML models, LLMs, and AI products or features.
• Strong understanding of the ML development cycle, from data and training to evaluation, deployment, and monitoring.
• Solid machine learning fundamentals and LLM knowledge, including agentic workflows, and you keep up with the latest developments in the field.
• Track record of applying LLMs, or agentic workflows to ML evaluation and quality.
• Experience in data curation, dataset creation, and dataset quality, including iterative improvement workflows that deliver high-quality datasets tailored to specific products.
• Experience designing, implementing, maintaining, or optimizing ML evaluation infrastructure at scale.
• Experience designing evaluation metrics based on the model, the product, and its use cases.
• Experience driving end-to-end evaluation and quality for AI products.
• Fluency in quality reporting, presenting quality data to both technical and non-technical audiences.
• Strong communication skills and experience working with and driving cross-functional partners.
• Proficient in Python and/or Swift or comparable languages.
• Bachelor's degree in Computer Science, Data Science, Machine Learning, or a related field, or equivalent experience.

Preferred Qualifications

• Knowledge of the software development life cycle and software quality assurance methodologies.
• Experience in objective image and video quality evaluation and metrics.
• Knowledge of color science and/or experience with subjective image or video quality review.
• Passion for creative applications; experience as a creator (photo, video, music, or design) is a plus.
• Excitement about learning new technology and applying it to practical problems.
• Familiarity with iOS and macOS.

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $142,300 and $263,300, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan. You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant

At Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.

Learn about accessibility in Apple’s workplace

Learn about reasonable accommodations for job applicants

Apple accepts applications to this posting on an ongoing basis.