GPU Kernel Engineer – AI Inference & CUDA Optimizations

FriendliAI Inc.; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Flexible working hours

Daily meals / snacks provided

Health insurance

Startup equity

Collaborative team environment

Job summary

FriendliAI Inc. is seeking a GPU Kernel Engineer to design, build, and optimize low-level compute kernels powering our GPU-accelerated AI inference platform. You will write GPU code for NVIDIA and AMD GPUs, focusing on high throughput and low latency across models on Hugging Face.

You will collaborate with the engine and platform teams to deploy optimizations, benchmark performance, and push advances in mixed-precision kernels, quantization, and multi-modal workloads.

Qualifications

• 3+ years of GPU programming, HPC, or performance-critical systems.
• Bachelor’s or Master’s degree in Computer Science, Computer Engineering, Electrical Engineering, or related field.
• Strong proficiency in CUDA for NVIDIA GPUs or ROCm/HIP for AMD GPUs.
• Deep understanding of GPU architecture and memory hierarchy.
• Proficiency in C++ and GPU profiling/optimisation.

Responsibilities

• Design, implement, and optimize high-performance GPU kernels for AI inference.
• Develop and maintain GPU code in CUDA and C++, including low-level assembly when needed.
• Implement reduced-precision and quantized kernels (FP8/FP4) for low-latency/high-throughput inference.
• Benchmark and ensure cross-vendor parity between NVIDIA and AMD hardware.
• Contribute to internal GPU libraries and tune performance-critical components.
• Accelerate multi-modal model pipelines and explore next-generation GPU features.

Skills

GPU programming

CUDA

C++

Performance tuning

HPC concepts

Profiling tools

Quantization

Education

Bachelor’s or Master’s in CS/CE/EE

Tools

CUTLASS

Triton

ROCm

CUDA toolkit

Job description

FriendliAI Inc. is seeking a GPU Kernel Engineer to design, build, and optimize low-level compute kernels powering our GPU-accelerated AI inference platform. You will write GPU code for NVIDIA and AMD GPUs, focusing on high throughput and low latency across models on Hugging Face.

You will collaborate with the engine and platform teams to deploy optimizations, benchmark performance, and push advances in mixed-precision kernels, quantization, and multi-modal workloads.