Senior AI Evaluation & Quality Systems Scientist

Apple, Inc.; Seattle (WA); In loco; USD 142.000 - 263.000; Tempo pieno

Vantaggi offerti da questo lavoro

Medical and dental coverage

Employee stock programs

Tuition reimbursement

Descrizione del lavoro

Apple is seeking a Senior Applied Scientist to join the Human-centered AI, ML Data Quality Operations team in Seattle. You will help build scalable evaluation systems to ensure the data powering AI/ML meets high standards of accuracy, consistency, and relevance.

You will prototype, validate, and ship ground-truth generation pipelines, real-time monitoring, and calibration methods for LLM evaluators, collaborating with ML teams, annotators, and downstream users to drive quality across products

Competenze

• 5+ years of industry experience in applied science or ML with production-grade evaluation pipelines.
• Experience designing ground truth generation pipelines across varied task types and modalities.
• Experience building real-time monitoring or anomaly detection systems for ML pipelines.
• Strong Python and ML frameworks with production experience building, deploying, and monitoring LLM-based pipelines.

Mansioni

• Design scalable ground truth generation pipelines across varied task types, annotation modalities, and cold start conditions.
• Build real-time monitoring systems that detect drift, distribution shifts, and quality degradation across live evaluation and annotation pipelines.
• Design calibration frameworks that re-anchor LLM evaluators against human-verified gold sets.
• Build root-cause analysis tools that surface disagreement patterns between automated and human judgments.
• Partner with ML teams, LLM-as-a-Judge developers, annotators to ground design decisions in real feedback.
• Communicate findings clearly to both technical and non-technical stakeholders.

Conoscenze

Applied ML pipelines

Python

ML frameworks

Real-time monitoring

Ground truth pipelines

Stakeholder communication

Formazione

MS/PhD in CS/ML/Statistics

Strumenti

TensorFlow

PyTorch

MLflow

Descrizione del lavoro

Apple is seeking a Senior Applied Scientist to join the Human-centered AI, ML Data Quality Operations team in Seattle. You will help build scalable evaluation systems to ensure the data powering AI/ML meets high standards of accuracy, consistency, and relevance.

You will prototype, validate, and ship ground-truth generation pipelines, real-time monitoring, and calibration methods for LLM evaluators, collaborating with ML teams, annotators, and downstream users to drive quality across products