LLM Inference Optimization Engineer

GMI Cloud; Mountain View (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

GMI Cloud, a fast-growing AI infrastructure company, is hiring a Machine Learning Engineer, LLM Optimization to build a world-leading inference optimization team. You will drive research, validation, and productionization of advanced optimization techniques to boost latency, throughput, and cost efficiency across the inference platform.

You will focus on B200-first optimization with H200 evolution, collaborating with platform and infra teams to turn new ideas into measurable customer-ready

Qualifications

• Experience with LLM inference systems and performance optimization.
• Familiar with inference metrics and tradeoffs (throughput, latency, memory).
• Hands-on with GPU-based inference and model serving architectures.

Responsibilities

• Drive frontier research and engineering in LLM inference optimization for high performance.
• Develop optimization strategies for large-scale LLM serving across runtimes and production platforms.
• Advance quantization, speculative decoding, KV cache, and memory optimization techniques.

Skills

LLM inference

Python

GPU inference

Performance profiling

System optimization

Benchmarks

observability

Cross-functional collaboration

Tools

SGLang

vLLM

TensorRT-LLM

Triton

Job description

GMI Cloud, a fast-growing AI infrastructure company, is hiring a Machine Learning Engineer, LLM Optimization to build a world-leading inference optimization team. You will drive research, validation, and productionization of advanced optimization techniques to boost latency, throughput, and cost efficiency across the inference platform.

You will focus on B200-first optimization with H200 evolution, collaborating with platform and infra teams to turn new ideas into measurable customer-ready