Senior LLM Inference Engineer: Performance & Optimization

Confidential; United States; On-site; USD 180,000 - 240,000; Full time

Job summary

Confidential in the United States seeks a senior engineer to own performance optimization for production LLM inference, focusing on latency, throughput, and cost across kernels and serving engines. You will profile GPU performance, apply quantization and batching strategies, and extend serving stacks like vLLM, TensorRT-LLM, and Triton.

You will collaborate with model and platform teams to push new architectures from works to fast, while targeting multi-GPU and accelerator-rich deployments.

Qualifications

• Deep experience optimizing deep-learning inference in production.
• Hands-on GPU programming and performance engineering (CUDA or equivalent).
• Fluency with modern LLM serving stacks (vLLM / TensorRT-LLM / SGLang / Triton).
• A track record of measurable performance wins (latency / throughput / cost).
• Strong systems fundamentals and a profiling-first mindset.

Responsibilities

• Optimize LLM inference for latency, throughput, and cost — at the kernel and serving-engine level.
• Profile and tune GPU performance (CUDA, TensorRT-LLM); apply quantization, speculative decoding, and batching strategies.
• Get the most out of serving frameworks like vLLM, SGLang, and Triton — and extend them where they fall short.
• Optimize across hardware targets where relevant (NVIDIA and other accelerators).
• Partner with model and platform teams to take new architectures from "works" to "fast".

Skills

Deep learning inference

GPU programming

Performance engineering

Profiling

CUDA

Tools

TensorRT-LLM

vLLM

SGLang

Triton

Job description

Confidential in the United States seeks a senior engineer to own performance optimization for production LLM inference, focusing on latency, throughput, and cost across kernels and serving engines. You will profile GPU performance, apply quantization and batching strategies, and extend serving stacks like vLLM, TensorRT-LLM, and Triton.

You will collaborate with model and platform teams to push new architectures from works to fast, while targeting multi-GPU and accelerator-rich deployments.