Senior LLM Inference QA & Automation Engineer

Apple, Inc.

Seattle (WA)

On-site

USD 140,000 - 190,000

Full time

6 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Apple is seeking a senior AI systems engineer to own LLM inference correctness, consistency, and stability in production. You will design and build automation frameworks, evaluation methodologies, and qualification strategies for models, prompts, serving stacks, and infrastructure changes before they reach users.

You will lead qualification for generative AI workloads, write production-quality code in Python/C++/Swift, and collaborate with ML engineers, inference engineers, and platform teams to

Qualifications

  • 5+ years in hardware/software development lifecycle processes.
  • Ability to own LLM inference quality in production and define qualification strategies.

Responsibilities

  • Design and develop qualification frameworks for LLM inference at data center scale.
  • Build the LLM inference output-correctness automation framework for large-scale deployment.
  • Architect end-to-end test/qualification strategy for distributed ML serving systems.
  • Design harnesses for validation of ML/LLM services with mocking and synthetic traffic.
  • Gate releases in CI: ensure model/weight/prompt/runtime changes are evaluated pre-release.
  • Create regression suites tracking quality across versions over time.
  • Validate tool-use, function-calling, and multi-turn agentic workflows for correctness.
  • Develop safety, refusal, and policy-compliance coverage across releases.
  • Probe AI-specific risks: prompt injection, data leakage, hallucinations, latency.
  • Build adversarial/robustness tests, long-context and ambiguous inputs.
  • Qualify inference-optimization techniques for output parity with full-precision references.
  • Validate output equivalence across serving backends and accelerators.
  • Perform comprehensive testing including performance, fault-injection, reliability.
  • Develop AI QA tooling for automated test generation and defect triage.
  • Define metrics, logging, tracing, and observability for production monitoring.
  • Apply log analysis to root-cause failures in complex LLM serving systems.
  • Conduct regression and integration testing across platforms including iOS and macOS.
  • Develop production AI infrastructure software in Python/C++/Swift with good modularity and performance.

Skills

LLM inference experience
Automation framework design
Quality engineering
Statistical ML evaluation
Production systems ownership
Python
C++
Swift
Data-center scale testing

Tools

Python
C++
Swift
Splunk
CI/CD tooling

Job description

Apple is seeking a senior AI systems engineer to own LLM inference correctness, consistency, and stability in production. You will design and build automation frameworks, evaluation methodologies, and qualification strategies for models, prompts, serving stacks, and infrastructure changes before they reach users.

You will lead qualification for generative AI workloads, write production-quality code in Python/C++/Swift, and collaborate with ML engineers, inference engineers, and platform teams to

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AIML - Sr. LLM Inference Quality Engineer
AIML - Sr. LLM Inference Quality Engineer

Apple, Inc. • Seattle (WA)

On-site
USD 140,000 - 190,000
AIML LLM Inference Quality Engineering Manager
AIML LLM Inference Quality Engineering Manager

Apple, Inc. • Seattle (WA)

On-site
USD 196,000 - 338,000
Medical & dental coverage
Stock programs & RSUs
Education reimbursement
+1
AI/ML Quality Engineer — LLM Evaluation & QA Automation
AI/ML Quality Engineer — LLM Evaluation & QA Automation

INSPYR Solutions • Cupertino (CA)

Hybrid
USD 91,000 - 105,000
Remote-friendly schedule
Senior ML Engineer — Open-Domain QA & LLM Research
Senior ML Engineer — Open-Domain QA & LLM Research

Apple, Inc. • Seattle (WA)

On-site
USD 175,000 - 309,000
Stock options
Relocation assistance
Comprehensive benefits
+1
Senior ML Platform Engineer – Gen AI & LLM
Senior ML Platform Engineer – Gen AI & LLM

Apple, Inc. • Seattle (WA)

On-site
USD 175,000 - 309,000
Medical and dental coverage
Retirement benefits
Employee stock programs
+1
Senior ML Engineer — Generative AI & LLM Leader
Senior ML Engineer — Generative AI & LLM Leader

Apple, Inc. • San Francisco (CA)

On-site
USD 185,000 - 325,000
Discretionary bonuses or commission
Relocation
Employee stock programs
Senior AI/ML Engineer - GenAI & LLMs for Global Marketing
Senior AI/ML Engineer - GenAI & LLMs for Global Marketing

Apple, Inc. • Austin (TX)

On-site
USD 180,000 - 260,000
Senior ML Engineer: LLMs, Agent Systems & Simulation
Senior ML Engineer: LLMs, Agent Systems & Simulation

Apple, Inc. • Cupertino (CA)

On-site
USD 150,000 - 278,000
Medical and dental coverage
Employee stock programs
Educational reimbursement (tuition)
Senior QA Automation Engineer - AI & LLM Test Frameworks
Senior QA Automation Engineer - AI & LLM Test Frameworks

Apple, Inc. • Cupertino (CA)

On-site
USD 140,000 - 190,000
ML Engineer – LLMs, NLP & Generative AI for Search
ML Engineer – LLMs, NLP & Generative AI for Search

Apple, Inc. • Cupertino (CA)

On-site
USD 180,000 - 260,000