ML Evaluation & Quality Engineer – Creativity Apps

Apple, Inc.; San Diego (CA); On-site; USD 142,000 - 263,000; Full time

Job summary

Apple in San Diego is seeking a Machine Learning Engineer focused on evaluation and quality for Creativity Apps. You will define how to measure ship-worthiness, design evaluation strategies, curate datasets, and build scalable evaluation infrastructure across ML features that power creative products.

You will collaborate with ML engineering, product, design, and infrastructure teams to translate use cases into metrics, tests, and quality signals that reflect creators’ real experiences.

Qualifications

• 5+ years of industry experience in ML or software engineering, with a focus on ML/AI model evaluation and quality.
• Experience with evaluation methodology for ML models, LLMs, and AI products or features.
• Strong understanding of the ML development cycle, from data and training to evaluation, deployment, and monitoring.
• Solid ML fundamentals and LLM knowledge, including agentic workflows, and you keep up with the latest developments in the field.
• Experience designing evaluation metrics based on the model, the product, and its use cases.

Responsibilities

• Define evaluation strategy and metrics for ML models, LLMs, and AI features, tailored to the model, product, and use cases.
• Curate and create evaluation datasets, measure and improve dataset quality, and run iterative improvement loops so that datasets stay representative of each product.
• Design, build, maintain, and optimize evaluation infrastructure at scale, with reproducible runs, baselines, and comparisons between model versions.
• Apply ML, LLMs, and agentic workflows to the evaluation itself, including automated judging, test generation, and analysis.
• Drive end-to-end evaluation and quality of AI features, from model checkpoints to shipping features, triaging issues to resolution.
• Report quality clearly, turning evaluation results into ship/no-ship recommendations for technical and non-technical audiences.

Skills

ML model evaluation

Cross-functional collaboration

Python

Swift

Communication

Education

Bachelor's degree in CS/Data Science/ML or related

Tools

Python

Swift

TensorFlow

PyTorch

Job description

Apple in San Diego is seeking a Machine Learning Engineer focused on evaluation and quality for Creativity Apps. You will define how to measure ship-worthiness, design evaluation strategies, curate datasets, and build scalable evaluation infrastructure across ML features that power creative products.

You will collaborate with ML engineering, product, design, and infrastructure teams to translate use cases into metrics, tests, and quality signals that reflect creators’ real experiences.