Machine Learning Engineer, AI/ML Model Quality

Apple, Inc.; San Diego (CA); On-site; USD 175,000 - 309,000; Full time

Benefits offered by this job

Medical & dental

Retirement benefits

Employee stock programs

Tuition reimbursement

Bonuses or commissions

Relocation

Job summary

Apple, Inc. seeks a Machine Learning Engineer, AI/ML Model Quality, to own data health for ML and GenAI across Wallet, Payments, and Commerce.

Build validation frameworks, observability metrics, and telemetry to ensure models are trained, evaluated, and deployed on trusted data for hundreds of millions of users. You will work with Data Engineering, Privacy, and Legal teams to calibrate and measure robustness, monitor data quality, and translate signals into actions for engineering and product

Qualifications

• Bachelor's degree or higher in a quantitative field or equivalent hands-on experience.
• 3+ years of production ML health metrics, monitoring, and observability.
• 3+ years in ML/data science roles focusing on data quality, model evaluation, or observability.
• Proficiency in Python (Pandas, NumPy, Scikit-learn) and SQL for data analysis.
• Experience with distributed computing (PySpark/Spark).
• Solid understanding of statistics, data drift, hypothesis testing, and CSP.
• Experience with GenAI/LLM systems and telemetry instrumentation.
• Strong communication to translate data findings to engineers and stakeholders.

Responsibilities

• Curate gold-standard ground-truth data for model evaluation and validation.
• Audit training data for bias and data drift with analytical checks.
• Define telemetry across GenAI workflows and monitor quality signals.
• Detect model degradation in production with privacy-preserving methods.
• Define and own data quality observability metrics and dashboards.
• Develop real-time dashboards and reporting for stakeholders.

Skills

Python

Pandas

NumPy

Scikit-learn

PyTorch

SQL

PySpark

Spark

Education

Bachelor's degree

MS/PhD in ML or related

Tools

Tableau

Databricks

Superset

LangSmith

MLflow

Weights & Biases

Spark

PySpark

Job description

San Diego, California, United States Software and Services

Would you like to contribute to Machine Learning and Generative AI technologies? Are you passionate about the integrity of the data that powers AI systems at scale? Do you believe that trustworthy data is the foundation of every great model? We truly believe it is!We are defining what exceptional data quality looks like for machine learning across Wallet, Payments, and Commerce. As a Machine Learning Engineer, AI/ML Model Quality, you will build and maintain intelligent systems, validation frameworks, synthetic data generation models and monitoring pipelines that keep our data ecosystem healthy — ensuring that every model we build is trained, evaluated, and deployed on data we can trust. Your work sits at the foundation of every ML feature that reaches hundreds of millions of users.You'll work at the intersection of machine learning, statistical rigor and production systems, collaborating closely with Data Engineering, Privacy, and Legal teams. This unique opportunity puts you at the center of ML and AI quality — owning the health of training and validation datasets, defining and analyzing observability metrics, building models to calibrate & measure robustness, and leading telemetry analysis across GenAI workflows — ensuring Apple's financial features are built on the highest-quality data, whether powering conventional ML models or the latest generative AI systems.

Description

The ideal candidate is detail-obsessed who understands that model quality starts long before training — it starts with the data. You have strong statistical instincts, know how silent degradation and data drift manifest in production systems, and can translate raw quality signals into insights that drive real decisions.You will own the health of the data ecosystem that underpins ML and GenAI features across Wallet, Payments, and Commerce — building validation frameworks, defining observability metrics, and leading telemetry analysis that keeps every model trained, evaluated, and monitored on data teams can trust. Your work sits at the foundation of every ML feature that reaches hundreds of millions of users.

Responsibilities

• Curate, analyze, and maintain gold-standard ground-truth datasets for model evaluation and continuous validation across both ML and GenAI systems.
• Audit training data for systemic bias and fairness gaps prior to model deployment; establish ongoing analytical checks to catch bias introduced by data drift over time.
• Define and analyze telemetry across GenAI workflows, tracking quality signals such as output coherence, latency, task completion rates, and regression patterns.
• Design mechanisms to detect degradation of models in production in a privacy preserving manner.
• Define and own ML observability metrics — model performance, output distributions, training-serving skew, silent degradation and feature drift — translating raw production signals into actionable insights for engineering and product teams.
• Design and develop observability dashboards and reporting workflows that give stakeholders a consistent, real-time view of model health across both conventional ML and GenAI systems.
• Identify degradation patterns and domain-specific failure modes in GenAI systems through systematic telemetry analysis, translating findings into concrete recommendations for model and data teams.
• Define, track, and report key data quality metrics — completeness, accuracy, timeliness, validity for engineering and leadership audiences.

Minimum Qualifications

• A Bachelor's degree with exceptional hands-on experience in ML/AI model quality or applied research or a M.S or Ph.D in Machine Learning, Computer Science, Data Science, Statistics, Mathematics, Engineering, or a related quantitative field is strongly preferred.
• 3+ years of experience in defining and tracking ML model health metrics in production — model performance monitoring, feature drift detection, and observability instrumentation.
• 3+ years of experience in Machine Learning, data science or a closely related analytical role, with a strong focus on data quality, model evaluation, or ML observability in production environments.
• Proficiency in Python (Pandas, NumPy, Scikit-learn), common ML frameworks like PyTorch and SQL for complex data analysis, metric creation, and validation.
• Experience querying and analyzing large-scale datasets using distributed computing frameworks (e.g., PySpark, Spark, or distributed SQL).
• Solid understanding of statistical methods — hypothesis testing, distribution analysis, data drift detection, and statistical process control.
• Experience with GenAI or LLM systems, including common quality failure modes, output evaluation approaches, and telemetry instrumentation.
• Strong communication skills — ability to translate complex data quality findings and model health risks into clear, actionable insights for both engineering and non-technical stakeholders.

Preferred Qualifications

• Experience with data visualization and dashboarding tools (e.g., Tableau, Apache Superset, Databricks) to present complex ML telemetry.
• Familiarity with LLM evaluation frameworks (e.g. LangSmith) or techniques like LLM-as-a-judge.
• Experience with Bayesian or causal graph-based approaches to synthetic data generation.
• Familiarity with confidence calibration techniques and uncertainty quantification.
• Experience with ML monitoring or observability platforms (e.g., MLflow, Weights & Biases, or equivalent).
• Experience working with privacy-constrained data or under regulatory compliance frameworks (GDPR, DMA).
• Background in financial services, fintech, or consumer payment products.

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $175,000 and $308,500, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple's Employee Stock Purchase Plan.

You’ll also receive benefits including:

• Comprehensive medical and dental coverage
• Retirement benefits
• A range of discounted products and free services
• For formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition
• Discretionary bonuses or commission payments
• Relocation

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics.

At Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.