Staff ML Performance Engineer - GPU & Inference

Modal; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

Modal is building an infrastructure layer for AI and is seeking strong engineers to optimize ML systems for performance at scale. You will contribute to Modal’s container runtime and open-source projects, pushing language and diffusion models toward higher throughput and lower latency.

The role emphasizes working with Torch, TensorRT, CUDA, and NVIDIA GPU architectures to maximize efficiency, while exploring low-level OS foundations to improve performance and reliability.

Qualifications

• 5+ years of experience writing high-quality, high-performance code.
• Experience with Torch, high-level ML frameworks, and inference engines (vLLM or TensorRT).
• Familiarity with Nvidia GPU architecture and CUDA.
• Experience with ML performance engineering and boosting GPU performance (e.g., debugging SM occupancy, rewriting algorithms, reducing host overhead).
• Nice-to-have: familiarity with low-level OS foundations (Linux kernel, file systems, containers).

Responsibilities

• Build ML systems that perform at scale and low latency.
• Contribute to open-source projects and Modal’s container runtime.
• Advance language and diffusion models toward higher throughput.

Skills

5+ years of experience writing high‑on

Tools

Torch

TensorRT

CUDA

Linux kernel

Containers

Job description

Modal is building an infrastructure layer for AI and is seeking strong engineers to optimize ML systems for performance at scale. You will contribute to Modal’s container runtime and open-source projects, pushing language and diffusion models toward higher throughput and lower latency.

The role emphasizes working with Torch, TensorRT, CUDA, and NVIDIA GPU architectures to maximize efficiency, while exploring low-level OS foundations to improve performance and reliability.