Machine Learning Engineer, AI/ML Model Quality

Apple, Inc.; New York (NY); On-site; USD 185,000 - 325,000; Full time

Benefits offered by this job

Medical and dental coverage

Retirement benefits

Employee stock plan

Discounted products and services

Educational expense reimbursement

Job summary

Apple, Inc. seeks a data-centric ML engineer to strengthen data and model quality across Wallet, Payments, and Commerce experiences. You will build validation frameworks, synthetic data models, and monitoring pipelines to safeguard data from training to deployment.

You will own observability metrics and telemetry analysis for traditional ML as well as GenAI workflows, collaborating with Data Engineering, Privacy, and Legal teams to ensure robust, compliant systems.

Qualifications

• Bachelor's degree plus hands-on ML/AI model quality or applied research experience, or MS/PhD in ML/CS/DS/Statistics/Math/Engineering (strongly preferred)
• 3+ years defining and tracking ML model health metrics in production, including model performance monitoring, feature drift detection, and observability instrumentation
• 3+ years in ML, data science, or related analytical role focused on data quality, model evaluation, or ML observability in production
• Proficiency in Python with Pandas, NumPy, Scikit-learn, plus SQL for complex data analysis, metrics creation and validation
• Experience with large-scale data using PySpark, Spark, or distributed SQL
• Strong understanding of statistics (hypothesis testing, distribution analysis, data drift, SPC)
• Experience with GenAI/LLM systems, QA and telemetry instrumentation
• Strong communication skills to translate risks into actionable insights

Responsibilities

• Build and maintain validation frameworks, synthetic data generation models, and monitoring pipelines to keep data ecosystem healthy from training through deployment
• Own end-to-end data ecosystem health by defining observability metrics and driving telemetry analysis for conventional ML and GenAI workflows
• Collaborate with Data Engineering, Privacy, and Legal teams
• Define, implement, and analyze observability metrics for ML system quality and reliability
• Build models to calibrate and measure robustness
• Lead telemetry analysis across GenAI workflows

Skills

ML model health metrics

Observability

Telemetry analysis

Python

Pandas

NumPy

Scikit-learn

SQL

PySpark

Spark

GenAI / LLM

Data visualization

Education

Bachelor's degree

MS or PhD in ML / CS / Data Science

Tools

Python

Pandas

NumPy

Scikit-learn

SQL

PySpark

Spark

Tableau

Apache Superset

Databricks

LangSmith

MLflow

Weights & Biases

Job description

Use machine learning expertise to strengthen data and model quality across Apple's Wallet, Payments, and Commerce experiences.

Responsibilities

• Build and maintain validation frameworks, synthetic data generation models, and monitoring pipelines to keep the data ecosystem healthy from training through evaluation and deployment
• Own end-to-end data ecosystem health by defining observability metrics and driving telemetry analysis for conventional ML and generative AI workflows
• Collaborate with Data Engineering, Privacy, and Legal teams
• Define, implement, and analyze observability metrics for ML system quality and reliability
• Build models to calibrate and measure robustness
• Lead telemetry analysis across GenAI workflows

Requirements

• Bachelor's degree plus exceptional hands‑on ML/AI model quality or applied research experience, or M.S. or Ph.D. in Machine Learning, Computer Science, Data Science, Statistics, Mathematics, Engineering, or a related quantitative field (strongly preferred)
• 3+ years defining and tracking ML model health metrics in production, including model performance monitoring, feature drift detection, and observability instrumentation
• 3+ years in Machine Learning, data science, or a closely related analytical role with a focus on data quality, model evaluation, or ML observability in production environments
• Proficiency in Python including Pandas, NumPy, and Scikit‑learn, plus SQL for complex data analysis, metric creation, and validation
• Experience querying and analyzing large‑scale datasets using distributed computing (e.g., PySpark, Spark, or distributed SQL)
• Solid understanding of statistical methods such as hypothesis testing, distribution analysis, data drift detection, and statistical process control
• Experience with GenAI/LLM systems, including common quality failure modes, output evaluation approaches, and telemetry instrumentation
• Strong communication skills to translate complex data quality and model health risks into clear, actionable insights for engineering and non‑technical stakeholders

Preferred Qualifications

• Experience with data visualization and dashboarding tools (e.g., Tableau, Apache Superset, Databricks) to present ML telemetry
• Familiarity with LLM evaluation frameworks (e.g., LangSmith) or techniques such as LLM-as-a-judge
• Experience with Bayesian or causal graph-based approaches to synthetic data generation
• Familiarity with confidence calibration techniques and uncertainty quantification
• Experience with ML monitoring or observability platforms (e.g., MLflow, Weights & Biases, or equivalent)
• Experience working with privacy-constrained data or under regulatory compliance frameworks (e.g., GDPR, DMA)
• Background in financial services, fintech, or consumer payment products

Technologies

• Python, Pandas, NumPy, Scikit‑learn
• PyTorch
• SQL
• PySpark, Spark
• Tableau, Apache Superset, Databricks
• LangSmith
• MLflow, Weights & Biases

Benefits

• Comprehensive medical and dental coverage
• Retirement benefits
• Discounted products and free services
• Reimbursement for certain educational expenses, including tuition
• Opportunity to become an Apple shareholder via Apple's discretionary employee stock programs
• Eligible for discretionary restricted stock unit awards
• Ability to purchase Apple stock at a discount through voluntary participation in the Employee Stock Purchase Plan

Location: New York, NY (onsite)

Salary: USD 184,700 - 324,800 per year

Minimum Experience: 3 years