GPU Kernel Engineer for Fast AI Training & Inference

River AI; Palo Alto (CA); On-site; USD 200,000 - 420,000; Full time

Benefits offered by this job

Equity

Visa sponsorship

Relocation assistance

Comprehensive health insurance

Job summary

River AI in Palo Alto, California is seeking exceptional GPU kernel engineers to accelerate large-model training and inference. You will own performance-critical operations, including attention, matrix multiplication, and low-precision compute, and collaborate with researchers and systems engineers to push the speed and efficiency of our stack.

You will design fast kernels, optimize memory and tiling, develop FP8/FP4 mixed-precision work, and validate correctness through rigorous benchmarks.

Qualifications

• Bachelor's degree in Computer Science, Computer Engineering, or equivalent practical experience.
• Experience optimizing GPU kernels with CUDA, Triton, CUTLASS, CuTe, or comparable tools.
• Strong understanding of GPU architecture, memory hierarchies, and parallel execution.
• Proficiency in C++ and Python.
• Strong foundations in linear algebra, floating-point arithmetic, and numerical computing.
• Strong debugging and profiling skills, with a collaborative approach to engineering.

Responsibilities

• Build fast GPU kernels for attention, matrix multiplication, expert routing, and related operations.
• Optimize memory access, tiling, and synchronization to make efficient use of GPU hardware.
• Develop FP8, FP4, and mixed-precision kernels while preserving numerical correctness.
• Accelerate fine-tuning and RL through optimized adapters, backward passes, and fused operations.
• Profile real workloads and integrate improvements into training and inference runtimes.
• Build reproducible benchmarks that verify correctness, gradients, and performance.

Skills

CUDA

GPU architecture

Python

C++

Education

Bachelor's degree in Computer Science/Engineering

Tools

Triton

CUTLASS

CuTe

PyTorch

Job description

River AI in Palo Alto, California is seeking exceptional GPU kernel engineers to accelerate large-model training and inference. You will own performance-critical operations, including attention, matrix multiplication, and low-precision compute, and collaborate with researchers and systems engineers to push the speed and efficiency of our stack.

You will design fast kernels, optimize memory and tiling, develop FP8/FP4 mixed-precision work, and validate correctness through rigorous benchmarks.