Inference Systems Engineer - Large-Scale GPU & Kubernetes

Luma; United States; On-site; USD 195,000 - 345,000; Full time

Job summary

Luma is seeking a systems engineer to own large-scale model serving—integrating new architectures into the inference engine, scaling deployments across thousands of machines, and keeping GPU fleets busy while meeting SLOs.

You will work cross-functionally with research, engineering, and infrastructure to optimize model efficiency, build tooling to measure inference job lifetimes, and automate testing to maximize uptime and reliability across clusters and providers.

Qualifications

• Strong Python and system-architecture skills.
• Experience deploying models with PyTorch, Hugging Face, vLLM, SGLang, TensorRT-LLM, or similar.
• Experience with queues, scheduling, traffic control, and fleet management at scale.
• Experience with Linux, Docker, and Kubernetes, and with orchestration, deployment, and scheduling.
• Familiarity with Redis and S3-compatible storage.

Responsibilities

• Ship new model architectures by integrating them into the inference engine.
• Collaborate across research, engineering, and infrastructure to optimize model efficiency and deployments.
• Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows.
• Automate, test, and maintain inference services for maximum uptime and reliability.
• Manage and optimize inference workloads across clusters and hardware providers, and scale deployments across thousands of machines.
• Build scheduling systems that use expensive GPU resources optimally while meeting SLOs, and maintain CI/CD for model checkpoints and SDKs.

Skills

Python

System architecture

Model serving

Distributed systems

Linux

Docker

Kubernetes

Redis

S3 compatible storage

PyTorch

Hugging Face

vLLM

SGLang

TensorRT-LLM

Tools

PyTorch

Hugging Face

vLLM

SGLang

TensorRT-LLM

Docker

Kubernetes

Redis

S3-compatible storage

Job description

Luma is seeking a systems engineer to own large-scale model serving—integrating new architectures into the inference engine, scaling deployments across thousands of machines, and keeping GPU fleets busy while meeting SLOs.

You will work cross-functionally with research, engineering, and infrastructure to optimize model efficiency, build tooling to measure inference job lifetimes, and automate testing to maximize uptime and reliability across clusters and providers.