Inference Performance Engineer — GPU Kernels & Systems

Acceler8 Talent; San Francisco (CA); On-site; USD 180,000 - 220,000; Full time

Job summary

Acceler8 Talent is seeking a Member of Technical Staff, Inference Performance to accelerate AI model inference across heterogeneous hardware. You will drive optimizations in latency, throughput, and memory usage, and push the serving stack with batching, caching, and quantization techniques.

The role involves developing CUDA/HIP/Triton kernels, deploying on accelerator clusters, and benchmarking models under real production workloads to improve efficiency and cost effectiveness.

Qualifications

• Evidence of work in high-performance AI inference or model-serving systems.
• Experience with GPU kernel development using CUDA, HIP, or Triton.
• Knowledge of quantization, speculative decoding, batching, or KV-cache optimization.
• Familiarity with PyTorch, vLLM, SGLang, TensorRT-LLM, or similar frameworks.
• Experience with GPU, accelerator, HPC, or distributed-compute infrastructure.
• Strong background in low-level performance profiling and systems optimization.
• Ability to operate latency-sensitive systems in production.

Responsibilities

• Profile inference latency, throughput, and memory usage.
• Improve serving engines through batching, caching, quantization, and speculative decoding.
• Develop and tune CUDA/HIP/Triton kernels.
• Operate heterogeneous accelerator clusters.
• Benchmark models and hardware under production workloads.
• Build observability and reliability into the serving platform.
• Debug performance across models, runtimes, kernels, networking, and hardware.

Skills

Inference performance

GPU kernel development

Quantization

PyTorch

TensorRT-LLM

CUDA

HIP

Triton

Model serving

Profiling

Low-level optimization

Distributed compute

Job description

Acceler8 Talent is seeking a Member of Technical Staff, Inference Performance to accelerate AI model inference across heterogeneous hardware. You will drive optimizations in latency, throughput, and memory usage, and push the serving stack with batching, caching, and quantization techniques.

The role involves developing CUDA/HIP/Triton kernels, deploying on accelerator clusters, and benchmarking models under real production workloads to improve efficiency and cost effectiveness.