Senior Software Engineer, Large-Scale Model Inference

Luma; Greater London; On-site; GBP 147,000 - 261,000; Full time

Job summary

Luma seeks a Senior Systems Engineer to own how its models are served, integrating new architectures into the inference engine and scaling deployments across thousands of machines.

This is large-scale inference systems work, including scheduling, fleet management, deployment pipelines, and reliability across clusters and hardware providers. You’ll work across research, engineering, and infrastructure to optimize efficiency.

Qualifications

• Strong Python and system-architecture skills.
• Experience deploying models with PyTorch, Hugging Face, vLLM, SGLang, or TensorRT-LLM.
• Experience with queues, scheduling, traffic control, and fleet management at scale.
• Experience with Linux, Docker, and Kubernetes, and with orchestration, deployment, and scheduling.
• Familiarity with Redis and S3-compatible storage.

Responsibilities

• Ship new model architectures by integrating them into the inference engine.
• Collaborate across research, engineering, and infrastructure to optimize model efficiency and deployments.
• Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows.
• Automate, test, and maintain inference services for maximum uptime and reliability.
• Manage and optimize inference workloads across clusters and hardware providers, and scale deployments across thousands of machines.
• Build scheduling systems that use expensive GPU resources optimally while meeting SLOs, and maintain CI/CD for model checkpoints and SDKs.

Skills

Python

System architecture

Model serving at scale

distributed systems

Tools

Docker

Kubernetes

Redis

S3-compatible storage

Job description

Luma seeks a Senior Systems Engineer to own how its models are served, integrating new architectures into the inference engine and scaling deployments across thousands of machines.

This is large-scale inference systems work, including scheduling, fleet management, deployment pipelines, and reliability across clusters and hardware providers. You’ll work across research, engineering, and infrastructure to optimize efficiency.