LLM Inference Optimization Engineer

GMI Cloud; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

GMI Cloud is building the leading inference optimization solution and the most advanced token platform in the global token market. We are hiring world-class Machine Learning Engineers to make GMI the new industry benchmark for LLM serving performance, cost efficiency, and production reliability.

This role focuses on frontier research, validation, and productionization of advanced inference optimization techniques, with close collaboration across platform and infrastructure teams to deliver

Qualifications

• Experience with LLM inference systems and GPU performance optimization.
• Understanding of inference metrics (throughput, latency, goodput) and tradeoffs.
• Familiarity with modern serving stacks (SGLang, vLLM, TensorRT-LLM, NVIDIA Dynamo, Triton).

Responsibilities

• Drive frontier research and engineering in LLM inference optimization across focus tracks while contributing to the full optimization stack.
• Develop next-generation optimization strategies for large-scale LLM serving across model execution and runtime systems.
• Advance state-of-the-art techniques in quantization, speculative decoding, KV cache/memory management, and PD disaggregation.
• Collaborate with platform, infrastructure, and product teams to translate ideas into measurable gains in latency, throughput, and cost efficiency.

Skills

LLM inference systems

GPU performance optimization

Inference metrics

Serving stacks

Experimentation

Education

PhD in related areas

Tools

SGLang

vLLM

TensorRT-LLM

NVIDIA Dynamo

Triton

CUDA

Job description

GMI Cloud is building the leading inference optimization solution and the most advanced token platform in the global token market. We are hiring world-class Machine Learning Engineers to make GMI the new industry benchmark for LLM serving performance, cost efficiency, and production reliability.

This role focuses on frontier research, validation, and productionization of advanced inference optimization techniques, with close collaboration across platform and infrastructure teams to deliver