Inference Systems Engineer for Scalable Multimodal AI

Luma AI; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Luma AI in San Francisco is building cutting-edge multimodal AI systems. We seek a seasoned Platform Engineer to ship new model architectures into our inference engine and optimize deployments across clusters.

You will collaborate with research, engineering and infra, develop scalable scheduling, CI/CD pipelines, and tooling to measure and ensure uptime for inference workloads across thousands of GPUs. Proficiency in Python, Linux, Docker, Kubernetes, and model deployment frameworks is required;

Qualifications

• Strong Python and system architecture skills.
• Experience deploying models with PyTorch, Huggingface, vLLM, SGLang, or tensorRT-LLM.
• Experience with Linux, Docker, and Kubernetes.
• Familiarity with queues, scheduling, and fleet management at scale.
• Bonus: RDMA networking, large-scale ML systems (>100 GPUs), FFmpeg.

Responsibilities

• Ship new model architectures by integrating them into our inference engine.
• Collaborate across research, engineering and infrastructure to optimize deployments.
• Build internal tooling to measure, profile, and track inference jobs.
• Automate, test and maintain inference services for maximum uptime and reliability.
• Optimize deployment workflows to scale across thousands of machines.
• Manage inference workloads across clusters and hardware providers.
• Build scheduling systems to efficiently leverage GPU resources while meeting SLOs.
• Develop CI/CD pipelines for processing/optimizing model checkpoints and platform components.

Skills

Python

System architecture

Model deployment

PyTorch

Huggingface

vLLM

SGLang

tensorRT-LLM

Queues & scheduling

Linux

Docker

Kubernetes

RDMA

High scale ML

FFmpeg

Job description

Luma AI in San Francisco is building cutting-edge multimodal AI systems. We seek a seasoned Platform Engineer to ship new model architectures into our inference engine and optimize deployments across clusters.

You will collaborate with research, engineering and infra, develop scalable scheduling, CI/CD pipelines, and tooling to measure and ensure uptime for inference workloads across thousands of GPUs. Proficiency in Python, Linux, Docker, Kubernetes, and model deployment frameworks is required;