Remote Engineering Lead, Inference Optimization

Venice.ai, Inc.; Northern (KY); Hybrid; USD 270,000 - 330,000; Full time

Job summary

Venice AI, Inc. is seeking an Engineering Lead to drive inference performance and scalable GPU optimization for private, high-volume AI workloads.

You will build and lead the Inference Optimization Team, optimize across architectures, and push latency, throughput, and cost per token to new levels while shaping the technical strategy for Venice’s private AI stack. Reporting to the Head of Engineering, the role combines hands-on development with people leadership in a fast-moving startup

Qualifications

• 8+ years in performance optimization or HPC.
• 5+ years leading engineering teams.
• Proficiency in Python, Rust, or Go.
• Hands-on experience with a production LLM inference engine (e.g. vLLM, SGLang).
• Demonstrated experience with LLM inference optimization techniques: batching, KV cache management, quantization, CUDA graphs.

Responsibilities

• Own Venice’s technical strategy for inference performance.
• Recruit and lead the Inference Optimization Team at Venice.
• Optimize Venice's GPU infrastructure across architectures (e.g. H200s, B300s).
• Improve latency, throughput, and cost per token for LLM inference workloads.
• Build reproducible benchmarking harnesses across inference engines to identify the optimal engine and strategy per workload.
• Work with our inference routing system to optimize multivariate inference load-balancing algorithms.
• Evaluate emerging inference optimization techniques and hardware viability for Venice's stack.

Skills

GPU optimization

Python

Rust

Go

HPC

Leadership

Production LLM

CUDA

Tools

vLLM

SGLang

CUDA

Triton

Nsight

torch.compile

Job description

Venice AI, Inc. is seeking an Engineering Lead to drive inference performance and scalable GPU optimization for private, high-volume AI workloads.

You will build and lead the Inference Optimization Team, optimize across architectures, and push latency, throughput, and cost per token to new levels while shaping the technical strategy for Venice’s private AI stack. Reporting to the Head of Engineering, the role combines hands-on development with people leadership in a fast-moving startup