ML Engineer - Foundation Models Infra (Low Latency)

Apple; Seattle (WA); On-site; USD 170,000 - 230,000; Full time

Job summary

Apple’s Foundation Model Services team builds the frameworks, services, and tools that run Apple's largest foundation models in production. You’ll work with product teams to deploy real-time model serving, and you’ll partner with researchers to prototype inference for state-of-the-art architectures.

You will create tooling to understand bottlenecks across hardware and use cases. Expect to write high-quality code, move quickly in a fast-evolving field, and grow your impact as the system scales to

Qualifications

• Bachelor's degree or higher in CS or related field.
• 2+ years building and shipping production software or ML systems.
• Proficiency in Go or Python.
• Experience deploying and operating services on cloud platforms using containers and Kubernetes/Docker.
• 5+ years of experience in ML technologies (LLMs, NLP, IR, statistics).
• Experience building or operating high-throughput, low-latency services.
• Strong communication and cross-team collaboration.

Responsibilities

• Collaborate with product teams to deliver production-grade model serving in real time.
• Prototype and implement inference for cutting-edge model architectures with researchers.
• Build tools to diagnose performance bottlenecks across hardware and use-cases.

Skills

Go

Python

Cloud platforms

Kubernetes

Communication

Team collaboration

Education

Bachelor's degree in Computer Science or related technical field

Tools

Docker

Kubernetes

NVIDIA TensorRT-LLM

DeepSpeed

Job description

Apple’s Foundation Model Services team builds the frameworks, services, and tools that run Apple's largest foundation models in production. You’ll work with product teams to deploy real-time model serving, and you’ll partner with researchers to prototype inference for state-of-the-art architectures.

You will create tooling to understand bottlenecks across hardware and use cases. Expect to write high-quality code, move quickly in a fast-evolving field, and grow your impact as the system scales to