LLM Inference Systems Engineer

Acceler8 Talent; San Francisco (CA); Hybrid; USD 160,000 - 260,000; Full time

Job summary

Acceler8 Talent is seeking a Machine Learning Engineer to optimize LLM inference performance in production. The role focuses on squeezing more performance from GPUs, distributed systems, and multi-node setups with Kubernetes-backed infrastructure.

You will work on high-performance infrastructure, profiling bottlenecks, and tuning batching, quantisation, and caching to scale AI workloads efficiently.

Qualifications

• You'll come from an ML systems, distributed computing, HPC or performance engineering background and be comfortable working close to the hardware.
• Strong Python and/or C++, PyTorch and GPU computing experience are key.
• Experience with CUDA, NCCL, Triton, vLLM, TensorRT-LLM or SGLang would be particularly relevant.
• You should be comfortable reasoning about how GPU utilisation, memory bandwidth, communication overhead and model architecture interact to impact inference performance.

Responsibilities

• Architect high-performance infrastructure for LLM inference
• Optimise latency, throughput and cost across production workloads
• Scale inference across multi-GPU and multi-node environments
• Improve GPU scheduling, allocation and utilisation
• Profile bottlenecks across compute, memory and networking
• Optimise batching, quantisation, KV caching and parallelism
• Scale GPU workloads across Kubernetes infrastructure

Skills

Python

C++

PyTorch

CUDA

NCCL

Triton

vLLM

TensorRT-LLM

SGLang

GPU

Tools

Kubernetes

Job description

Acceler8 Talent is seeking a Machine Learning Engineer to optimize LLM inference performance in production. The role focuses on squeezing more performance from GPUs, distributed systems, and multi-node setups with Kubernetes-backed infrastructure.

You will work on high-performance infrastructure, profiling bottlenecks, and tuning batching, quantisation, and caching to scale AI workloads efficiently.