Senior ML Engineer — Foundation Model Inference at Scale

Apple Inc.; New York (NY); On-site; USD 185,000 - 325,000; Full time

Benefits offered by this job

Employee stock programs

Bonus opportunities

Relocation assistance

Education reimbursement

Job summary

Apple Inc. in New York City, NY, is seeking a Sr. Machine Learning Engineer to join the Foundation Model Inference team within Cloud OS & Inference.

You will help deploy high-performance, private inference stacks powering Siri and Apple AI across Apple services with low latency. You'll bridge research and production, owning inference efficiency, hardware/software codesign, and scalable tooling. The role requires hands-on ML infrastructure experience and a track record in large-scale systems,

Qualifications

• 5+ years leading complex, ambiguous technical projects.
• Hands-on experience with LLM inference stacks.
• Working knowledge of GPU or TPU programming concepts.
• Proficiency with PyTorch, JAX, or TensorFlow.
• Experience building and operating high-throughput services at large distributed scale.
• Proficiency deploying applications on cloud platforms (AWS, GCP, or equivalent) using Kubernetes and Docker.
• BS in Computer Science, Machine Learning, Artificial Intelligence, Data Science, or a related field.

Responsibilities

• Partner with the Foundation Model Research team and external partners to optimize inference for the latest model architectures across language, vision, and speech.
• Design and ship production-grade inference systems serving millions of customers in real time.
• Build profiling tools and simulators to identify and resolve performance bottlenecks across different hardware configurations and use cases.
• Drive technical decisions on high-throughput, low-latency serving at supercomputing scale.
• Mentor and grow engineers across the organization.

Skills

LLM inference

Leadership / project leadership

GPU/TPU concepts

PyTorch/JAX/TensorFlow

Distributed high-throughput systems

Kubernetes & Docker deployment

Education

BS in Computer Science / related field

MS in CS / ML / AI / DS

Tools

Kubernetes

Docker

CUDA

OpenAI Triton

TensorRT-LLM

vLLM

Job description

Apple Inc. in New York City, NY, is seeking a Sr. Machine Learning Engineer to join the Foundation Model Inference team within Cloud OS & Inference.

You will help deploy high-performance, private inference stacks powering Siri and Apple AI across Apple services with low latency. You'll bridge research and production, owning inference efficiency, hardware/software codesign, and scalable tooling. The role requires hands-on ML infrastructure experience and a track record in large-scale systems,