LLM Inference Engineer - GPU Systems & Optimization

GMI Cloud, Inc; San Francisco (CA); On-site; USD 180,000 - 280,000; Full time

Job summary

GMI Cloud, Inc seeks world-class Machine Learning Engineers to push the frontier of LLM inference optimization. You will spearhead research and implementation across quantization, speculative decoding, KV cache, and memory management to deliver measurable improvements in latency, throughput, and cost efficiency for GPU-backed AI workloads.

Collaborate with platform and infrastructure teams to translate ideas into scalable production systems, contribute to open-source projects, and drive

Qualifications

• Strong hands-on experience with LLM inference systems and performance optimization.
• Experience with modern serving stacks (SGLang, vLLM, TensorRT-LLM, NVIDIA Dynamo, Triton).
• Proven ability to design benchmarks, interpret results, and draw actionable conclusions.
• Excellent communication to explain tradeoffs to engineers and stakeholders.

Responsibilities

• Drive research and engineering in LLM inference optimization across focus areas (Speculative Decoding, Quantization, PD disaggregation, KV cache/memory).
• Develop optimization strategies for large-scale model serving on GPUs, targeting primary platforms (B200) and evolving (H200).
• Advance state-of-the-art methods in quantization, speculative decoding, and memory management.
• Build scalable optimization frameworks, tooling, and benchmark infrastructure to stay ahead of industry trends.
• Productionize ideas into customer workloads and publish results to open-source communities.

Skills

LLM inference systems

Performance optimization

Benchmarking

Experimentation

Profiling/Observability

Communication of tradeoffs

Education

PhD in related areas

Tools

SGLang

vLLM

TensorRT-LLM

NVIDIA Dynamo

Triton

Job description

GMI Cloud, Inc seeks world-class Machine Learning Engineers to push the frontier of LLM inference optimization. You will spearhead research and implementation across quantization, speculative decoding, KV cache, and memory management to deliver measurable improvements in latency, throughput, and cost efficiency for GPU-backed AI workloads.

Collaborate with platform and infrastructure teams to translate ideas into scalable production systems, contribute to open-source projects, and drive