Inference Systems Engineer: Scale GPU Fleets & Kubernetes

Luma; Greater London; On-site; GBP 110,000 - 170,000; Full time

Job summary

Luma is building large-scale inference systems and needs a systems engineer to own model serving within the inference engine. You will scale deployments across thousands of machines, optimize GPU utilization, and ensure reliability across clusters and hardware providers.

Ideal candidates have strong Python and system-architecture skills, experience with Linux, Docker, Kubernetes, and model deployment frameworks such as PyTorch, Hugging Face, vLLM, or TensorRT-LLM. Equal opportunity employer.

Qualifications

• Experience building scalable inference systems.
• Experience deploying models with PyTorch, Hugging Face, vLLM, SGLang or similar.
• Strong Linux, Docker, and Kubernetes experience.
• Familiarity with distributed scheduling and fleet management at scale.
• Proficient Python and systems background.

Responsibilities

• Own how Luma’s models get served and integrate architectures into the inference engine.
• Scale deployments across thousands of machines and manage workloads across clusters and hardware providers.
• Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows.
• Automate, test, and maintain inference services for uptime and reliability.
• Build scheduling systems that optimize GPU resources while meeting SLOs and maintain CI/CD for model checkpoints.
• Collaborate with research, engineering, and infrastructure to improve efficiency.

Skills

Python

System Architecture

Model deployment

Linux

Docker & Kubernetes

Redis & S3 storage

Tools

PyTorch

Hugging Face

vLLM

SGLang

TensorRT-LLM

Job description

Luma is building large-scale inference systems and needs a systems engineer to own model serving within the inference engine. You will scale deployments across thousands of machines, optimize GPU utilization, and ensure reliability across clusters and hardware providers.

Ideal candidates have strong Python and system-architecture skills, experience with Linux, Docker, Kubernetes, and model deployment frameworks such as PyTorch, Hugging Face, vLLM, or TensorRT-LLM. Equal opportunity employer.