Hands-on Tech Lead: Inference Platform & Scaling

lumalabs-ai; San Francisco (CA); On-site; USD 230,000 - 350,000; Full time

Job summary

Luma is hiring a Tech Lead Manager to guide the inference platform team that governs the serving stack across GPUs and clusters. The role blends hands-on engineering with leadership, focusing on performance, reliability, and cost efficiency to enable rapid production of large models.

You will set technical direction, own roadmap, and mentor engineers while collaborating with research, product, and infrastructure to push new model architectures into production.

Qualifications

• 8+ years of engineering experience in large-scale distributed systems or ML infrastructure.
• Experience running inference platforms at scale across multiple clusters or clouds.
• Technical leadership experience with hands-on work and growth mindset.
• Deep expertise in LLM and foundation-model serving engines (vLLM, SGLang, TensorRT-LLM).
• Strong skills in serving-performance tools: batching, KV-cache, quantization, decoding.
• Proficiency in Python and PyTorch; Kubernetes at scale.
• Experience with queues, scheduling, and fleet management at scale.

Responsibilities

• Hands-on design, build, and debugging in the serving stack while leading the team.
• Hire, coach, and grow inference engineers; build operational culture.
• Set roadmap for serving platform: routing, autoscaling, observability, deployment.
• Own platform SLOs, latency, availability, GPU utilization, and cost per generation.
• Collaborate with research to ship models to production and integrate RL loops.

Skills

Distributed systems

ML infrastructure

Leadership

Hands-on coding

Python

PyTorch

Kubernetes

System design

Tools

TensorRT-LLM

vLLM

SGLang

Kubernetes

Job description

Luma is hiring a Tech Lead Manager to guide the inference platform team that governs the serving stack across GPUs and clusters. The role blends hands-on engineering with leadership, focusing on performance, reliability, and cost efficiency to enable rapid production of large models.

You will set technical direction, own roadmap, and mentor engineers while collaborating with research, product, and infrastructure to push new model architectures into production.