Staff/Sr. Machine Learning Engineer, Foundation Models - AI, Search & Knowledge Platforms

Socket.dev; Santa Clara (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Apple seeks a leader for the Foundation Model Inference Team to optimize inference for cutting-edge model architectures and to build production-grade solutions that serve millions of users in real time.

You will mentor engineers, collaborate with product teams, and develop tooling to identify bottlenecks across GPUs and cloud infrastructure, enabling low-latency AI at scale.

Qualifications

• 5+ years leading complex, ambiguous projects.
• Experience with LLM inference stacks.
• CUDA programming familiarity.
• Proficiency with PyTorch or TensorFlow.
• Experience with high-throughput services at large scale.
• Experience on Cloud (AWS/Azure) with Kubernetes and Docker.

Responsibilities

• Optimize inference for cutting-edge model architectures.
• Build production-grade solutions to serve models at scale.
• Build tools to understand bottlenecks in inference across hardware.
• Mentor and guide engineers in the organization.

Skills

Project leadership

LLM inference

CUDA

PyTorch

TensorFlow

Kubernetes

Docker

Cloud (AWS/Azure)

Python

Go

Transformers

OpenAI Triton

CUDA kernels

Education

BS in Computer Science, AI, ML, IR or related field

MS in Computer Science, AI, ML, IR or related field

Tools

TensorRT-LLM

vLLM

DeepSpeed

Nvidia Triton Server

CUDA

OpenAI Triton

Job description

We are Foundation Model Inference Team, within AI, Search & Knowledge Platform Technologies organization. Our team is responsible to build Inference stack to power Apple Intelligence. It builds frameworks, services and tools that power the largest Apple foundation models on servers. Our Infrastructure powers a wide gamut of services at Apple including Apple Search, Apple Music, AppleTV, AppStore, iMessages, Photos & Camera, Spotlight, Safari, Siri and upcoming ever exciting Apple products serving millions of queries every day with incredible low latencies, drawing every ounce of compute from our hardware. As part of this group, you will get a chance to bring Intelligence to billions of users across the world. You will have an opportunity to make difference in life of people by empowering them with AI. You will have a chance to work on optimizing billions of parameter language and vision and speech models using state of the art technologies and make it run at scale of Apple.

Description

Work along side Foundation Model Research team to optimize inference for cutting edge model architectures.Work closely with product teams to build Production grade solutions to launch models serving millions of customers in real time.Build tools to understand bottlenecks in Inference for different hardwares and use cases.Mentor and guide engineers in the organization.

Minimum Qualifications

• 5+ years of experience leading and driving complex, ambiguous projects.
• Experience with LLM inference stack
• Familiarity with GPU programming concepts using CUDA.
• Familiarity with one of the popular ML Frameworks like Pytorch, Tensorflow.
• Have experience with high throughput services particularly at supercomputing scale.
• Proficient with running applications on Cloud (AWS / Azure or equivalent) using Kubernetes, Docker etc.
• Familiar with one of the popular ML Frameworks like Pytorch, Tensorflow.
• BS in Computer Science, Artificial Intelligence, Machine Learning, Information Retrieval, Data Science or related field

Preferred Qualifications

• Proficient in building and maintaining systems written in modern languages (eg: Golang, Python)
• Familiar with fundamental Deep Learning architectures such as Transformers, Encoder/Decoder models.
• Familiarity with Nvidia TensorRT-LLM, vLLM, DeepSpeed, Nvidia Triton Server etc.
• Experience writing custom CUDA kernels using CUDA or OpenAI Triton.
• MS in Computer Science, Artificial Intelligence, Machine Learning, Information Retrieval, Data Science or related field.