LLM Inference Library Engineer - High-Performance AI

Jobot; San Francisco (CA); On-site; USD 175,000 - 250,000; Full time

Benefits offered by this job

Equity (startup)

Competitive compensation

Healthcare, vision, dental

Job summary

Jobot is seeking an engineer to build and maintain a high-performance inference library for modern AI models across diverse compute architectures. The role targets someone who understands how LLM inference works under the hood and aims to squeeze maximum performance from complex hardware.

We value experience in AI/ML infrastructure, CUDA/Rocm, and model-serving frameworks, with strong Python and C++/Rust skills. Equity, competitive pay, and excellent benefits are offered.

Qualifications

• Experience in AI/ML infrastructure or HPC software.
• Programming in Python and C++, Rust, or similar systems languages.
• Experience with LLM inference frameworks and model-serving infrastructure.
• Hands-on CUDA, ROCm, Triton or similar GPU programming.
• Understanding of transformer/LLM architectures and memory management.
• Experience benchmarking AI workloads across hardware environments.
• Familiarity with batching, attention, KV caching, quantization, and memory management.

Responsibilities

• Build and maintain a high-performance inference library for AI models.
• Optimize performance across diverse compute architectures.
• Work with CUDA/ROCm and related frameworks to accelerate workloads.
• Analyze and improve batching, KV caching, and memory usage.

Skills

AI/ML infrastructure

Python

C++

Rust

LLM inference

GPU programming

Performance benchmarking

Transformer/LLM architectures

Memory management

Kernel optimization

Tools

CUDA

ROCm

Triton

vLLM

TensorRT-LLM

SGLang

Job description

Jobot is seeking an engineer to build and maintain a high-performance inference library for modern AI models across diverse compute architectures. The role targets someone who understands how LLM inference works under the hood and aims to squeeze maximum performance from complex hardware.

We value experience in AI/ML infrastructure, CUDA/Rocm, and model-serving frameworks, with strong Python and C++/Rust skills. Equity, competitive pay, and excellent benefits are offered.