Software Engineer - GPU Kernel

FriendliAI

San Francisco (CA)

On-site

USD 120,000 - 150,000

Full time

14 days+

Application generator

Turn this role into an interview — a resume and cover letter built around what this employer wants.

Get past ATS filters

Benefits offered by this job

Flexible working hours

Daily lunch and dinner

Health check-up support

Competitive compensation

Startup equity

Health insurance

Job summary

FriendliAI is seeking a GPU Kernel Engineer in San Francisco to design and optimize GPU kernels for AI inference. This role requires expertise in CUDA, C++, and performance-critical systems. You will work on cutting-edge GPU technology and contribute to a highly collaborative, supportive work environment with competitive compensation and startup equity. Join us at the forefront of AI infrastructure innovation.

Qualifications

3+ years of experience in GPU programming.
Strong proficiency in CUDA for NVIDIA GPUs or ROCm/HIP for AMD GPUs.
Deep understanding of GPU architecture and tuning.

Responsibilities

Design, implement, and optimize high-performance GPU kernels for AI inference.
Develop and maintain GPU code in CUDA and C++.
Benchmark and ensure performance parity between NVIDIA and AMD hardware.

Skills

GPU programming

Performance-critical systems

CUDA proficiency

C++ proficiency

Performance tuning

Education

Bachelor’s or Master’s in Computer Science, Computer Engineering, Electrical Engineering

Tools

CUDA

C++

ROCm/HIP

Job description

About the job

FriendliAI is looking for a GPU Kernel Engineer to design, build, and optimize the low-level compute kernels that power our large-scale, GPU-accelerated AI inference platform. You will be delivering world-class inference speed across NVIDIA and AMD GPUs. With our recent $20M funding, we are scaling our team to meet market demand.

This is a deeply technical, high-impact role where you will write GPU code, implement advanced optimizations. As part of our engine team, you will contribute directly to the company’s proprietary inference engine which supports over 450,000 models on Hugging Face. You will work with the inventors of continuous batching and collaborate with the platform team to deploy your work into production.

Key Responsibilities

Design, implement, and optimize high-performance GPU kernels for AI inference (e.g., GEMM, attention, routing)
Develop and maintain GPU code in CUDA and C++, including low-level assembly when needed
Implement reduced-precision and quantized kernels (FP8/FP4) for low-latency or high-throughput inference
Benchmark and ensure cross-vendor performance parity between NVIDIA and AMD hardware
Contribute to internal GPU libraries and tune performance of performance-critical components
Accelerate multi-modal model pipelines
Investigate and integrate next-generation GPU features

Qualifications

3+ years of experience in GPU programming, HPC, or performance-critical systems
Bachelor’s or Master’s degrees in Computer Science, Computer Engineering, Electrical Engineering, or a related field
Strong proficiency in CUDA for NVIDIA GPUs or ROCm/HIP for AMD GPUs
Deep understanding of GPU architecture: warps, threads, memory hierarchy, synchronization, and latency-throughput trade-offs
Proficiency in C++
Experience with GPU profiling and performance tuning
Strong numerical background with understanding of precision trade-offs and quantization techniques

Preferred Experience

Experience optimizing transformer, multi-modal, or Mixture-of-Experts (MoE) architectures at the kernel level
Familiarity with the latest GPU libraries and frameworks (CUTLASS, Triton, …)
Inter-GPU communication programming experience
Open-source contributions related to GPU performance or ML acceleration
Research or conference presentations on GPU optimization, HPC, or numerical computing

Benefits