Software Engineer - AI Inference Engine

FriendliAI; San Francisco (CA); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Flexible working hours

Daily lunch and dinner provided; unlimited snacks and beverages

Health check-up support and top-tier equipment/hardware support

Supportive and highly collaborative work environment

Startup equity and health insurance

Job summary

FriendliAI in San Francisco is looking for an experienced Inference Engine Engineer to optimize GPU kernel performance for AI workloads. The position requires a strong background in GPU programming and collaboration with cloud infrastructure teams. The ideal candidate should have 5+ years of experience and expertise in Python, C++. This role offers flexible hours and various benefits, including competitive compensation and health insurance.

Qualifications

• 5+ years of experience in production or high-impact research environments.
• Production-level expertise in Python and C++.
• Experience developing machine learning frameworks or performance-critical runtime systems.
• Hands-on experience writing and optimizing GPU kernels.
• Experience working with generative AI models such as transformer and diffusion models.

Responsibilities

• Design and optimize custom GPU kernels for AI workloads.
• Contribute to the development of the kernel compiler and other core components.
• Collaborate with cloud and infrastructure engineers for end-to-end inference performance.
• Analyze performance bottlenecks and implement optimizations.
• Maintain production-grade performance infrastructure.

Skills

GPU programming

Python

C++

Machine learning frameworks

Performance optimization

Education

Bachelor's or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent

Tools

Machine learning compilers

Profiling tools

Job description

About the Job

We are seeking a highly technical Inference Engine Engineer to optimize the performance and efficiency of our core inference engine. In this role, you will focus on designing, implementing, and optimizing GPU kernels and supporting infrastructure for next-generation generative and agentic AI workloads. Your work will directly power the most latency-critical and compute-intensive systems deployed by our customers.

We are looking for an exceptional engineer with a strong foundation in GPU programming and compiler infrastructure. The ideal candidate enjoys pushing performance boundaries and has experience supporting production-scale machine learning applications.

Key Responsibilities

• Design and optimize custom GPU kernels for AI (e.g., transformer and diffusion) workloads

• Contribute to the development of FriendliAI’s kernel compiler, memory planner, runtime, and other core components.

• Collaborate with cloud and infrastructure engineers to ensure end-to-end inference performance

• Analyze performance bottlenecks across the software and hardware stack, and implement targeted optimizations

• Drive support for new model architectures and tensor compute patterns

• Maintain production-grade performance infrastructure, including profiling, benchmarking, and validation tools

Qualifications

• 5+ years of experience in production or high-impact research environments

• Production-level expertise in Python and C++

• Bachelor’s or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent

• Experience developing machine learning frameworks or performance-critical runtime systems

• Hands-on experience writing and optimizing GPU kernels

• Hands-on experience profiling GPU kernels

• Experience working with generative AI models such as transformer and diffusion models

Preferred Experience

• Experience developing machine learning compilers or code generation systems

• Familiarity with dynamic shape compilation, memory planning, and kernel fusion

• Contributions to inference engines, compilers, or high-performance numerical libraries

• Understanding of multi-GPU and distributed inference strategies

Benefits

• Flexible working hours

• Daily lunch and dinner provided; unlimited snacks and beverages

• Supportive and highly collaborative work environment

• Health check-up support and top-tier equipment/hardware support

• A front-row seat to the generative AI infrastructure revolution

• Competitive compensation, startup equity, health insurance, and other benefits.