Inference Systems Engineer: Scale GPU Fleets & Deployments

AItoolnavio; Union (NJ); On-site; USD 180,000 - 240,000; Full time

Job summary

Luma is seeking an experienced Systems Engineer to own large-scale model serving and inference infrastructure. You will design deployment pipelines, optimize GPU utilization, and scale fleets across data centers and cloud providers.

The role emphasizes reliability, performance, and collaboration with research and infra teams. You will work with Python, Kubernetes, Docker, and ML frameworks to maintain uptime and efficient inference workloads, while advancing CI/CD for model checkpoints and

Qualifications

• Proven experience deploying ML models at scale.
• Experience with model serving stacks and inference pipelines.
• Hands-on work with Kubernetes, Docker, and Linux in production.

Responsibilities

• Ship new model architectures by integrating them into the inference engine.
• Collaborate across research, engineering, and infrastructure to optimize model efficiency and deployments.
• Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows.
• Automate, test, and maintain inference services for maximum uptime and reliability.
• Manage and optimize inference workloads across clusters and hardware providers, and scale deployments across thousands of machines.
• Build scheduling systems that use expensive GPU resources optimally while meeting SLOs, and maintain CI/CD for model checkpoints and SDKs.

Skills

Python

System architecture

Scheduling at scale

Tools

PyTorch

Hugging Face

vLLM

SGLang

TensorRT-LLM

Linux

Docker

Kubernetes

Redis

S3-compatible storage

CUDA

FFmpeg

Job description

Luma is seeking an experienced Systems Engineer to own large-scale model serving and inference infrastructure. You will design deployment pipelines, optimize GPU utilization, and scale fleets across data centers and cloud providers.

The role emphasizes reliability, performance, and collaboration with research and infra teams. You will work with Python, Kubernetes, Docker, and ML frameworks to maintain uptime and efficient inference workloads, while advancing CI/CD for model checkpoints and