Senior ML Inference Engineer: High-Performance GPU Systems

Acceler8 Talent; San Francisco, Northern (CA, KY); Hybrid; USD 180,000 - 240,000; Full time

Job summary

Acceler8 Talent is recruiting an ML Inference Engineer for a Stanford-spun AI startup in San Francisco that is building an eight-figure revenue and growth trajectory. You will design, implement, and optimize the infrastructure powering large-scale LLM workloads and real-time model serving.

Ideal candidates combine Python/C++ proficiency with distributed systems experience, PyTorch expertise, and a passion for low-latency, GPU-accelerated inference at production scale.

Qualifications

• Experience with high-performance computing or distributed systems.
• Strong knowledge of Python and C++ for performance-sensitive stack.
• Experience with LLM inference and model serving.
• Experience optimizing GPU-heavy workloads and real-time serving.

Responsibilities

• Build the infrastructure that serves large-scale LLM workloads.
• Push the limits of latency, throughput and GPU efficiency.
• Design distributed inference across single and multi-GPU systems.
• Improve GPU scheduling, orchestration and resource utilisation.
• Profile and remove bottlenecks across compute, memory and networking.
• Scale production workloads across Kubernetes and GPU clusters.
• Make low-level architecture decisions where milliseconds matter.

Skills

Python

C++

Distributed systems

LLM inference

PyTorch

Tools

CUDA

NCCL

Triton

vLLM

TensorRT-LLM

SGLang

Job description

Acceler8 Talent is recruiting an ML Inference Engineer for a Stanford-spun AI startup in San Francisco that is building an eight-figure revenue and growth trajectory. You will design, implement, and optimize the infrastructure powering large-scale LLM workloads and real-time model serving.

Ideal candidates combine Python/C++ proficiency with distributed systems experience, PyTorch expertise, and a passion for low-latency, GPU-accelerated inference at production scale.