LLM Inference Optimization Engineer

GMI Cloud; Mountain View (CA); On-site; USD 170,000 - 230,000; Full time

Job summary

GMI Cloud is building the leading inference optimization solution and the most advanced token platform for the global token market, hiring world‑class ML engineers to push the boundaries of LLM serving performance, cost efficiency, and reliability.

You will drive research, validation, and productionization of cutting-edge inference optimization techniques, turning ideas into measurable improvements over baselines and contributing back to the community.

Qualifications

• Strong systems and infrastructure background with Linux, networking, observability and distributed systems.
• Experience deploying production ML platforms and serving stacks.
• Familiarity with LLM inference optimization and benchmark workflows.

Responsibilities

• Design and operate a reliable, repeatable experiment framework for LLM inference across H200 and B200 fleets.
• Build A/B testing infrastructure against real traffic.
• Optimize and land optimized inference solutions for customized inference scenarios.
• Build the foundation for agentic / automated inference optimization.
• Drive reliability and fault recovery of advanced inference solution toward high SLA targets.
• Build elastic GPU provisioning across hardware and spot machines to support production and experiments.
• Diagnose and fix long-tail distributed inference performance bugs.
• Collaborate with ML engineers to benchmark, test, and roll out optimizations.
• Engage with open-source community (NVIDIA Dynamo, NCCL, vLLM, SGLang, TensorRT-LLM).

Skills

Linux / distributed systems

Python

Go / C++ / Rust

GPU infrastructure

Experimentation platforms

Hardware/Software debugging

Tools

NVIDIA Dynamo

vLLM

TensorRT-LLM

NCCL

InfiniBand

RoCE

Job description

GMI Cloud is building the leading inference optimization solution and the most advanced token platform for the global token market, hiring world‑class ML engineers to push the boundaries of LLM serving performance, cost efficiency, and reliability.

You will drive research, validation, and productionization of cutting-edge inference optimization techniques, turning ideas into measurable improvements over baselines and contributing back to the community.