Senior ML Engineer, Foundation Models Inference — Cloud OS

Apple Inc.; Santa Clara, Northern (CA, KY); Hybrid; USD 185,000 - 325,000; Full time

Benefits offered by this job

Medical and dental coverage

Retirement benefits

Employee stock programs

Job summary

Apple Inc. is seeking a Sr. Machine Learning Engineer for the Foundation Models Inference team in Santa Clara, CA.

You will collaborate with research and external partners to bring cutting-edge model architectures from prototype to planetary-scale deployment, owning hard problems in inference efficiency and system design. You will build and optimize inference systems on Apple’s privacy-preserving PCC/CloudOS infrastructure, ensuring low latency and high reliability across billions of queries.

Qualifications

• 5+ years leading complex, ambiguous technical projects.
• Hands-on experience with LLM inference stacks.
• Working knowledge of GPU or TPU programming concepts.
• Proficiency with PyTorch, JAX, or TensorFlow.
• Experience building and operating high-throughput services at large distributed scale.
• Proficiency deploying applications on cloud platforms using Kubernetes and Docker.
• BS in Computer Science, Machine Learning, AI, Data Science, or related field.

Responsibilities

• Partner with the Foundation Model Research team and external partners to optimize inference for language, vision, and speech.
• Design and ship production-grade inference systems serving millions of customers in real time.
• Build profiling tools and simulators to identify and resolve performance bottlenecks across hardware configurations.
• Drive technical decisions on high-throughput, low-latency serving at supercomputing scale.
• Mentor and grow engineers across the organization.

Skills

LLM inference

Distributed systems

GPU/TPU programming

ML frameworks (PyTorch/JAX/TF)

Cloud platforms (AWS/GCP)

Education

BS in Computer Science / related field

MS in Computer Science / related field

Tools

Kubernetes

Docker

CUDA

Job description

Apple Inc. is seeking a Sr. Machine Learning Engineer for the Foundation Models Inference team in Santa Clara, CA.

You will collaborate with research and external partners to bring cutting-edge model architectures from prototype to planetary-scale deployment, owning hard problems in inference efficiency and system design. You will build and optimize inference systems on Apple’s privacy-preserving PCC/CloudOS infrastructure, ensuring low latency and high reliability across billions of queries.