Inference Systems Engineer (GPU/Cluster Scaling)

AItoolnavio; Greater London; Hybrid; GBP 100,000 - 180,000; Full time

Job summary

Luma is seeking a systems engineer to own large-scale model serving and inference pipelines. You will integrate new architectures into the inference engine, scale deployments across thousands of machines, and keep GPU fleets busy while meeting internal SLOs.

You’ll work on scheduling, fleet management, and reliable deployment pipelines across clusters, with a strong emphasis on efficiency and uptime. This is the systems side of ML, not pure modeling.

Qualifications

• Experience deploying models at scale.
• Strong Python and system design skills.
• Familiarity with Linux, Docker and Kubernetes.
• Experience with distributed systems and model serving toolchains.

Responsibilities

• Ship new model architectures by integrating them into the inference engine.
• Collaborate across research, engineering, and infrastructure to optimize model efficiency and deployments.
• Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows.
• Automate, test, and maintain inference services for maximum uptime and reliability.
• Manage and optimize inference workloads across clusters and hardware providers, and scale deployments across thousands of machines.
• Build scheduling systems that use expensive GPU resources optimally while meeting SLOs, and maintain CI/CD for model checkpoints and SDKs.

Skills

Python

System architecture

Model deployment

PyTorch

Hugging Face

vLLM

SGLang

TensorRT-LLM

Queues

Scheduling

Fleet management

Linux

Docker

Kubernetes

Redis

S3 storage

Tools

Docker

Kubernetes

Redis

S3 storage

Job description

Luma is seeking a systems engineer to own large-scale model serving and inference pipelines. You will integrate new architectures into the inference engine, scale deployments across thousands of machines, and keep GPU fleets busy while meeting internal SLOs.

You’ll work on scheduling, fleet management, and reliable deployment pipelines across clusters, with a strong emphasis on efficiency and uptime. This is the systems side of ML, not pure modeling.