AI Inference Kernel Optimization Engineer

Vidman AI; United States; Remote; USD 140,000 - 210,000; Full time

Job summary

Vidman AI is seeking a senior systems engineer to accelerate open-model inference performance by optimizing GPU kernels, memory usage, and production runtimes. Your work will be measured by latency reductions and token cost improvements.

You will operate across low-level performance engineering and deployed serving systems, collaborating with research and platform engineers to support new model architectures and deliver upstream changes.

Qualifications

• 5+ years in systems software or ML systems engineering with hands-on GPU work.
• Strong C++ and Python with deep CUDA/Triton/CUTLASS experience.
• Experience profiling hardware and interpreting low-level GPU behavior.
• Familiarity with modern inference runtimes like vLLM or TensorRT-LLM.
• Demonstrated kernel optimization with before-and-after measurements.

Responsibilities

• Build and tune kernels for attention, matrix multiplication, and normalization.
• Improve data movement through cache reuse, KV-cache management, and operator fusion.
• Implement low-precision inference paths, including INT4, FP8, and INT8, for available hardware.
• Profile GPU execution, inspect PTX or SASS where needed, and ship optimizations.
• Integrate improvements into inference runtimes and contribute upstream.
• Coordinate with research and platform engineers to support new model architectures.

Skills

C++ performance engineering

Python

CUDA/Triton/CUTLASS

GPU profiling

ML systems

Tools

CUDA

TensorRT-LLM

vLLM

Job description

Role overview

Improve the speed and serving cost of open-model inference by optimizing GPU kernels, memory use, and production runtimes. You’ll work across low-level performance engineering and deployed serving systems, with results measured in latency and token cost.

Responsibilities

• Build and tune kernels for operations such as attention, matrix multiplication, and normalization.
• Improve data movement through cache reuse, KV-cache management, and operator fusion.
• Implement low-precision inference paths, including INT4, FP8, and INT8, for the available hardware.
• Profile GPU execution, inspect PTX or SASS where needed, and ship optimizations based on findings.
• Integrate improvements into inference runtimes and contribute appropriate changes upstream.
• Coordinate with research and platform engineers to support new model architectures.

Requirements

• At least five years in systems software or ML systems engineering, including hands-on GPU performance work.
• Strong C++ and Python skills, with deep experience in at least one of CUDA, Triton, or CUTLASS.
• Ability to use hardware-level profiling tools and interpret low-level GPU execution.
• Familiarity with a modern inference runtime such as vLLM or TensorRT-LLM, and with transformer inference performance and memory behavior.
• Demonstrated kernel optimization work supported by before-and-after performance measurements.