LLM Inference Engineer: Peak-Performance AI Serving

Majestic Labs ai; Los Altos (CA); On-site; USD 210,000 - 350,000; Full time

Job summary

Majestic Labs is seeking an experienced engineer to own the end-to-end LLM serving stack on Majestic hardware. You will port leading frameworks like vLLM and SGLang to our accelerator and optimize them for peak throughput, latency, and scale.

You will collaborate with kernel, compiler, and hardware teams to push performance and reliability in production deployments, shaping the future of high-performance AI inference.

Qualifications

• 5+ years building or operating production LLM inference and serving systems.
• Hands-on work with vLLM, SGLang, TensorRT-LLM, Fireworks or similar, including scheduler and KV cache.
• Strong Python and C++, with fluency between the two.
• Understanding of transformer inference, KV cache, batching and latency concepts.
• Distributed inference experience across multiple devices.

Responsibilities

• The serving stack end to end — bring up and adapt a modern inference framework to Majestic hardware.
• The runtime hot path — batching, scheduler, KV cache, and prefill/decode disaggregation.
• Distributed inference at scale — tensor and pipeline parallelism across accelerators connected to CCL.
• Multi-modal pipeline — image, audio, and video preprocessing and integration.
• Inference-time techniques — speculative decoding, prefix caching, structured decoding.
• End-to-end performance — profile, benchmark, and drive improvements with kernel and compiler teams.

Skills

Python

C++

LLM inference

Distributed systems

Performance optimization

Problem solving

Tools

vLLM

SGLang

TensorRT-LLM

Fireworks

CCL

Job description

Majestic Labs is seeking an experienced engineer to own the end-to-end LLM serving stack on Majestic hardware. You will port leading frameworks like vLLM and SGLang to our accelerator and optimize them for peak throughput, latency, and scale.

You will collaborate with kernel, compiler, and hardware teams to push performance and reliability in production deployments, shaping the future of high-performance AI inference.