ML Quality & Evaluation Engineer for AI Features

Lex

British Columbia

On-site

CAD 168,000 - 325,000

Full time

9 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Comprehensive medical and dental cover
Retirement benefits
Discounted products and free services
Tuition reimbursement
Relocation assistance (potential)

Job summary

Apple’s Creativity Apps team seeks a Machine Learning Engineer focused on evaluating ML models and AI features in our creativity tools. You will define evaluation metrics, build evaluation datasets, and develop scalable infrastructure to ensure quality from model development to customer delivery.

You will collaborate with ML engineering, infrastructure, product, and design to translate use cases into robust evaluation methods and clear, actionable quality metrics for ship decisions.

Qualifications

  • 5+ years of industry experience in machine learning or software engineering, with a focus on ML/AI model evaluation and quality.
  • Experience with evaluation methodology for ML models, LLMs, and AI products or features.
  • Strong understanding of the ML development cycle, from data and training to evaluation, deployment, and monitoring.
  • Solid machine learning fundamentals and LLM knowledge, including agentic workflows, and you keep up with the latest developments in the field.
  • Track record of applying LLMs, or agentic workflows to ML evaluation and quality.
  • Experience in data curation, dataset creation, and dataset quality, including iterative improvement workflows that deliver high-quality datasets tailored to specific products.
  • Experience designing evaluation metrics based on the model, the product, and its use cases.
  • Experience driving end-to-end evaluation and quality for AI products.
  • Fluency in quality reporting, presenting quality data to both technical and non-technical audiences.
  • Strong communication skills and experience working with and driving cross-functional partners.
  • Proficient in Python and/or Swift or comparable languages.

Responsibilities

  • Define evaluation strategy and metrics for ML models, LLMs, and AI features tailored to the model and product.
  • Curate and create evaluation datasets, measure and improve dataset quality, and run iterative improvement loops.
  • Design, build, maintain, and optimize evaluation infrastructure at scale with reproducible runs and baselines.
  • Apply ML, LLMs, and agentic workflows to the evaluation to improve efficiency and accuracy.
  • Drive end-to-end evaluation and quality for AI features from checkpoints to shipping features.
  • Report quality clearly, turning evaluation results into ship/no-ship recommendations for technical and non-technical audiences.
  • Partner with cross-functional teams and navigate ambiguity with informed judgments.

Skills

Python
Swift
ML/AI model evaluation
communication

Education

Bachelor's degree in Computer Science
Data Science
Machine Learning

Job description

Apple’s Creativity Apps team seeks a Machine Learning Engineer focused on evaluating ML models and AI features in our creativity tools. You will define evaluation metrics, build evaluation datasets, and develop scalable infrastructure to ensure quality from model development to customer delivery.

You will collaborate with ML engineering, infrastructure, product, and design to translate use cases into robust evaluation methods and clear, actionable quality metrics for ship decisions.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Machine Learning Engineer, Evaluation and Quality, Creativity Apps
Machine Learning Engineer, Evaluation and Quality, Creativity Apps

Apple, Inc. • Vancouver

On-site
CAD 168,000 - 325,000
Comprehensive benefits
Stock programs
Relocation assistance
AI Quality Engineer: Prompt Testing & Evaluation
AI Quality Engineer: Prompt Testing & Evaluation

Rootly • Canada

On-site
CAD 110,000 - 160,000
Competitive compensation
Comprehensive medical, dental, and视s
Vacation and mental health days
+3
AI / ML QA Engineer
AI / ML QA Engineer

Infotek Consulting Inc. • Toronto

On-site
CAD 85,000 - 110,000
AI Quality Engineer
AI Quality Engineer

Centraprise • Vancouver

On-site
CAD 80,000 - 100,000
Software Development Engineer in Test, Full Stack, Creativity Apps
Software Development Engineer in Test, Full Stack, Creativity Apps

Apple, Inc. • Vancouver

On-site
CAD 119,000 - 231,000
Stock options
Relocation assistance
Education reimbursement
+1
AI Engineer
AI Engineer

Talents LGS • Montreal

On-site
CAD 90,000 - 150,000
Machine Learning Engineer
Machine Learning Engineer

Compunnel Inc. • Toronto

On-site
CAD 100,000 - 130,000
Senior ML Engineer: Production AI Leader & Mentor
Senior ML Engineer: Production AI Leader & Mentor

Compunnel Inc. • Toronto

On-site
CAD 100,000 - 130,000
Edge ML Engineer — Hardware-Driven AI Innovator
Edge ML Engineer — Hardware-Driven AI Innovator

Urban Ridge Supplies • Markham

On-site
CAD 114,000 - 164,000
AI Content Quality Evaluator
AI Content Quality Evaluator

Crossing Hurdles • Canada

On-site
CAD 60,000 - 80,000