LLM Inference Optimization Engineer - Frontier Performance

GMI Cloud, Inc; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

GMI Cloud, Inc is hiring world-class Machine Learning Engineers to advance LLM inference optimization leveraging GPUs and cutting-edge techniques. You will drive research, validation, and productionization of optimization strategies across NV platforms, with a focus on speed, efficiency, and scalability.

You will collaborate with platform and infrastructure teams, contribute to open-source projects, and help define recipes and benchmarks for industry-leading inference performance.

Qualifications

• Must have hands-on experience with LLM inference systems and performance optimization.
• Familiarity with inference metrics (TTFT, ITL, throughput, tail latency) and cost tradeoffs.
• Experience with modern serving stacks (SGLang, vLLM, TensorRT-LLM, NVIDIA Dynamo, Triton).

Responsibilities

• Drive frontier research and engineering in LLM inference optimization across specified tracks.
• Develop optimization strategies for large-scale LLM serving with B200 primary target and H200 ongoing.
• Collaborate with platform and infra teams to translate ideas into measurable performance gains.
• Publish results and engage with open-source communities (vLLM, SGLang, TensorRT-LLM, NVIDIA Dynamo).
• Build scalable optimization frameworks and benchmarking infrastructure.

Skills

LLM inference systems

Performance optimization

TTFT/ITL metrics

GPU-based inference

Serving stacks (SGLang, vLLM, TensorRT

Experimentation & benchmarks

Claude Code advanced usage

Technical communication

Education

PhD in related areas

Tools

SGLang

vLLM

TensorRT-LLM

NVIDIA Dynamo

Triton

Job description

GMI Cloud, Inc is hiring world-class Machine Learning Engineers to advance LLM inference optimization leveraging GPUs and cutting-edge techniques. You will drive research, validation, and productionization of optimization strategies across NV platforms, with a focus on speed, efficiency, and scalability.

You will collaborate with platform and infrastructure teams, contribute to open-source projects, and help define recipes and benchmarks for industry-leading inference performance.