GPU Kernel Engineer for AI Inference & Performance

FriendliAI Inc.

San Francisco (CA)

On-site

USD 120,000 - 150,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Flexible working hours
Daily lunch and dinner
Health check-up support
Competitive compensation
Startup equity
Health insurance

Job summary

FriendliAI is seeking a GPU Kernel Engineer in San Francisco to design and optimize GPU kernels for AI inference. This role requires expertise in CUDA, C++, and performance-critical systems. You will work on cutting-edge GPU technology and contribute to a highly collaborative, supportive work environment with competitive compensation and startup equity. Join us at the forefront of AI infrastructure innovation.

Qualifications

  • 3+ years of experience in GPU programming.
  • Strong proficiency in CUDA for NVIDIA GPUs or ROCm/HIP for AMD GPUs.
  • Deep understanding of GPU architecture and tuning.

Responsibilities

  • Design, implement, and optimize high-performance GPU kernels for AI inference.
  • Develop and maintain GPU code in CUDA and C++.
  • Benchmark and ensure performance parity between NVIDIA and AMD hardware.

Skills

GPU programming
Performance-critical systems
CUDA proficiency
C++ proficiency
Performance tuning

Education

Bachelor’s or Master’s in Computer Science, Computer Engineering, Electrical Engineering

Tools

CUDA
C++
ROCm/HIP

Job description

About the job

FriendliAI is looking for a GPU Kernel Engineer to design, build, and optimize the low-level compute kernels that power our large-scale, GPU-accelerated AI inference platform. You will be delivering world-class inference speed across NVIDIA and AMD GPUs. With our recent $20M funding, we are scaling our team to meet market demand.

This is a deeply technical, high-impact role where you will write GPU code, implement advanced optimizations. As part of our engine team, you will contribute directly to the company’s proprietary inference engine which supports over 600,000 models on Hugging Face. You will work with the inventors of continuous batching and collaborate with the platform team to deploy your work into production.

Key Responsibilities
  • Design, implement, and optimize high-performance GPU kernels for AI inference (e.g., GEMM, attention, routing)

  • Develop and maintain GPU code in CUDA and C++, including low-level assembly when needed

  • Implement reduced-precision and quantized kernels (FP8/FP4) for low-latency or high-throughput inference

  • Benchmark and ensure cross-vendor performance parity between NVIDIA and AMD hardware

  • Contribute to internal GPU libraries and tune performance of performance-critical components

  • Accelerate multi-modal model pipelines

  • Investigate and integrate next-generation GPU features

Qualifications
  • 3+ years of experience in GPU programming, HPC, or performance-critical systems

  • Bachelor’s or Master’s degrees in Computer Science, Computer Engineering, Electrical Engineering, or a related field

  • Strong proficiency in CUDA for NVIDIA GPUs or ROCm/HIP for AMD GPUs

  • Deep understanding of GPU architecture: warps, threads, memory hierarchy, synchronization, and latency-throughput trade-offs

  • Proficiency in C++

  • Experience with GPU profiling and performance tuning

  • Strong numerical background with understanding of precision trade-offs and quantization techniques

Preferred Experience
  • Experience optimizing transformer, multi-modal, or Mixture-of-Experts (MoE) architectures at the kernel level

  • Familiarity with the latest GPU libraries and frameworks (CUTLASS, Triton, …)

  • Inter-GPU communication programming experience

  • Open-source contributions related to GPU performance or ML acceleration

  • Research or conference presentations on GPU optimization, HPC, or numerical computing

Benefits
  • Flexible working hours

  • Daily lunch and dinner provided; unlimited snacks and beverages

  • Supportive and highly collaborative work environment

  • Health check-up support and top-tier equipment/hardware support

  • A front-row seat to the generative AI infrastructure revolution

  • Competitive compensation, startup equity, health insurance, and other benefits.

About FriendliAI

FriendliAI is building the world’s best AI inference platform that makes large language and multi-modal models fast, efficient, and deployable at scale. We power high-throughput, low-latency AI workloads for organizations worldwide and integrate directly with Hugging Face, giving developers instant access to over 600,000 open-source models.

We are a small, fast-moving team doing work that matters at one of the most exciting moments in the history of technology. With our world-class inference engine, we are building a platform that the AI industry can actually rely on.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Software Engineer – GPU Kernel San Francisco
Software Engineer – GPU Kernel San Francisco

FriendliAI Inc. • San Francisco (CA)

On-site
USD 180,000 - 240,000
Flexible working hours
Daily meals / snacks provided
Health insurance
+2
Senior GPU Inference Engine Engineer
Senior GPU Inference Engine Engineer

FriendliAI Inc. • San Francisco (CA)

On-site
USD 120,000 - 160,000
Flexible working hours
Daily lunch and dinner provided; unlimited snacks and beverages
Health check-up support and top-tier equipment/hardware support
+2
Software Engineer – AI Inference Engine San Francisco
Software Engineer – AI Inference Engine San Francisco

FriendliAI Inc. • San Francisco (CA)

On-site
USD 190,000 - 270,000
Flexible hours
Lunch/dinner provided
Collaborative environment
+4
Software Engineer – GPU Kernel
Software Engineer – GPU Kernel

FriendliAI • San Francisco (CA)

On-site
USD 120,000 - 150,000
Flexible working hours
Daily lunch and dinner
Health check-up support
+3
GPU Kernel Engineer – AI Inference & CUDA Optimizations
GPU Kernel Engineer – AI Inference & CUDA Optimizations

FriendliAI Inc. • San Francisco (CA)

On-site
USD 180,000 - 240,000
Flexible working hours
Daily meals / snacks provided
Health insurance
+2
Software Engineer - Senior Backend San Francisco
Software Engineer - Senior Backend San Francisco

FriendliAI Inc. • San Francisco (CA)

On-site
USD 180,000 - 250,000
Flexible working hours
Daily lunch and dinner provided
Health check-up support
Software Engineer – AI Inference Engine
Software Engineer – AI Inference Engine

FriendliAI • San Francisco (CA)

On-site
USD 120,000 - 160,000
Flexible working hours
Daily lunch and dinner provided; unlimited snacks and beverages
Health check-up support and top-tier equipment/hardware support
+2
Software Engineer – Cloud Infrastructure San Francisco
Software Engineer – Cloud Infrastructure San Francisco

FriendliAI Inc. • San Francisco (CA)

On-site
USD 210,000 - 270,000
Flexible hours
Meals provided daily
Collaborative culture
+2
GPU/Kernel Engineer
GPU/Kernel Engineer

Acceler8 Talent • Palo Alto (CA)

On-site
USD 180,000 - 240,000
Senior Inference Engine Engineer - GPU Kernel & ML Runtime
Senior Inference Engine Engineer - GPU Kernel & ML Runtime

FriendliAI Inc. • San Francisco (CA)

On-site
USD 190,000 - 270,000
Flexible hours
Lunch/dinner provided
Collaborative environment
+4