Senior Inference Engine Engineer - GPU Kernel & ML Runtime

FriendliAI Inc.; San Francisco (CA); On-site; USD 190,000 - 270,000; Full time

Benefits offered by this job

Flexible hours

Lunch/dinner provided

Collaborative environment

Health check-ups

Top-tier hardware support

Startup equity

Health insurance

Job summary

FriendliAI Inc. seeks a highly technical Inference Engine Engineer to optimize its core inference engine and deliver high-performance AI workloads.

You will design, implement, and optimize GPU kernels and supporting infrastructure critical for latency-sensitive applications. The ideal candidate has a strong background in GPU programming, compiler-like infrastructure, and production-scale ML systems, with experience transforming transformer or diffusion models into efficient, scalable

Qualifications

• 5+ years in production or high-impact research environments.
• Production-level Python and C++ experience.
• Experience with ML frameworks and performance-critical runtime systems.
• Hands-on GPU kernel development and profiling.
• Experience with transformer and diffusion models.

Responsibilities

• Design and optimize custom GPU kernels for AI workloads (transformer, diffusion).
• Contribute to kernel compiler, memory planner, runtime, and core components.
• Collaborate with cloud/infrastructure engineers for end-to-end inference performance.
• Analyze bottlenecks across software/hardware stack and implement optimizations.
• Drive support for new model architectures and tensor compute patterns.
• Maintain production-grade performance infrastructure, including profiling and benchmarking tools.

Skills

Python

C++

GPU kernel programming

Profiling GPU workloads

ML frameworks

Compiler infrastructure

Distributed inference

Performance optimization

Education

Bachelor's or Master's degree in CS/CE/EE or equivalent

Job description

FriendliAI Inc. seeks a highly technical Inference Engine Engineer to optimize its core inference engine and deliver high-performance AI workloads.

You will design, implement, and optimize GPU kernels and supporting infrastructure critical for latency-sensitive applications. The ideal candidate has a strong background in GPU programming, compiler-like infrastructure, and production-scale ML systems, with experience transforming transformer or diffusion models into efficient, scalable