Senior AI Kernel Engineer — Remote/Hybrid Inference

Modular; United States; Hybrid; USD 198,000 - 286,000; Full time

Benefits offered by this job

Amazing Team

World-class Benefits

Competitive Compensation

Team Building Events

Job summary

Modular is seeking a Senior AI Kernel Engineer to design and optimize high-performance kernels for large-scale AI inference on GPUs and custom accelerators. You will own performance-critical paths and drive architectural decisions for production-ready workloads.

You will collaborate with hardware, compiler, and AI systems teams to deliver scalable inference performance. This role offers an in-office Los Altos, CA option or remote work, with in-person onboarding.

Qualifications

• 5+ years of experience in performance-critical systems or kernel development.
• Strong proficiency in C/C++ and low-level programming.
• Extensive hands-on experience with GPU kernel programming (CUDA, HIP, or equivalent).
• Deep understanding of GPU architecture, including memory hierarchies, synchronization, and execution models.
• Proven track record of delivering measurable performance improvements in production systems.
• Strong problem-solving skills and ability to work independently on complex performance challenges.

Responsibilities

• Design, implement, and optimize performance-critical kernels for AI inference workloads (GEMM, attention, fusion).
• Lead kernel-level optimization across single-GPU, multi-GPU, and heterogeneous hardware environments.
• Make trade-offs between latency, throughput, memory footprint, and numerical precision.
• Drive adoption of new hardware features (Tensor Cores, asynchronous execution, advanced memory spaces).
• Analyze performance with profilers, hardware counters, and microbenchmarks; translate insights into concrete improvements.
• Collaborate with compiler and runtime teams to influence code generation, scheduling, and kernel fusion strategies.
• Review and mentor other engineers on kernel design and performance tuning.
• Contribute to long-term performance strategy and roadmaps for AI inference.

Skills

performance-critical systems/kernel

C/C++

GPU kernel programming

GPU architecture

production performance improvements

independent problem-solving

Job description

Modular is seeking a Senior AI Kernel Engineer to design and optimize high-performance kernels for large-scale AI inference on GPUs and custom accelerators. You will own performance-critical paths and drive architectural decisions for production-ready workloads.

You will collaborate with hardware, compiler, and AI systems teams to deliver scalable inference performance. This role offers an in-office Los Altos, CA option or remote work, with in-person onboarding.