Head of Inference Optimization & GPU Performance

Venice.ai; United States; Remote; USD 180,000 - 280,000; Full time

Job summary

Venice AI is seeking a senior performance and HPC specialist to own inference performance strategy, lead the Inference Optimization Team, and optimize GPU infrastructure for large-scale LLM workloads. You will drive latency and throughput improvements, build reproducible benchmarks, and assess new techniques and hardware for Venice’s private, secure AI platform.

We expect 8+ years in performance optimization, strong GPU knowledge, and hands-on experience with production LLM engines.

Qualifications

• 8+ years in performance optimization or HPC with deep GPU architecture knowledge.
• Experience leading engineering teams (5+ years).
• Proficiency in Python, Rust, or Go; bonus for C++/CUDA.
• Hands‑on experience with production LLM inference engines (vLLM, SGLang).
• Familiar with quantization, batching, and CUDA/torch.compile techniques.

Responsibilities

• Own Venice’s technical strategy for inference performance.
• Recruit and lead the Inference Optimization Team.
• Optimize GPU infrastructure across architectures (e.g., H200s, B300s).
• Improve latency, throughput and cost per token for LLM workloads.
• Build benchmarking harnesses across engines to pick the best setup.
• Collaborate on inference routing and load balancing algorithms.
• Evaluate new techniques and hardware for stack optimization.

Skills

Performance optimization

GPU architecture

Parallel programming

Python

Rust

Go

C++/CUDA

LLM inference engine experience

Tools

Nsight Systems

Nsight Compute

PyTorch Profiler

Job description

Venice AI is seeking a senior performance and HPC specialist to own inference performance strategy, lead the Inference Optimization Team, and optimize GPU infrastructure for large-scale LLM workloads. You will drive latency and throughput improvements, build reproducible benchmarks, and assess new techniques and hardware for Venice’s private, secure AI platform.

We expect 8+ years in performance optimization, strong GPU knowledge, and hands-on experience with production LLM engines.