ML Inference Systems Engineer: Scale GPU Deployments

Luma; United Kingdom; On-site; GBP 90,000 - 150,000; Full time

Job summary

Luma is building unified general intelligence that can generate, understand, and operate in the physical world. This role owns large-scale inference systems, integrating new architectures into the inference engine, and scaling deployments across thousands of machines.

You will work with research, engineering and infrastructure to optimize model efficiency, build tooling, automate inference services, and manage GPU resources with Kubernetes and CI/CD for model checkpoints.

Qualifications

• Strong Python and system-architecture skills.
• Experience deploying models with PyTorch, Hugging Face, vLLM, SGLang, TensorRT-LLM, or similar.
• Experience with queues, scheduling, traffic control, and fleet management at scale.
• Experience with Linux, Docker, and Kubernetes, and with orchestration, deployment, and scheduling.
• Familiarity with Redis and S3-compatible storage.

Responsibilities

• Ship new model architectures by integrating them into the inference engine.
• Collaborate across research, engineering, and infrastructure to optimize model efficiency and deployments.
• Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows.
• Automate, test, and maintain inference services for maximum uptime and reliability.
• Manage and optimize inference workloads across clusters and hardware providers, and scale deployments across thousands of machines.
• Build scheduling systems that use expensive GPU resources optimally while meeting SLOs, and maintain CI/CD for model checkpoints and SDKs.

Skills

Python

System architecture

PyTorch

Hugging Face

vLLM

SGLang

TensorRT-LLM

Scheduling at scale

Linux

Docker

Kubernetes

Redis

S3 storage

Tools

Docker

Kubernetes

Redis

S3 storage

Job description

Luma is building unified general intelligence that can generate, understand, and operate in the physical world. This role owns large-scale inference systems, integrating new architectures into the inference engine, and scaling deployments across thousands of machines.

You will work with research, engineering and infrastructure to optimize model efficiency, build tooling, automate inference services, and manage GPU resources with Kubernetes and CI/CD for model checkpoints.