Kernel Engineer — High-Performance AI Inference

Acceler8 Talent; Santa Clara (CA); On-site; USD 170,000 - 350,000; Full time

Benefits offered by this job

Equity / stock options

Medical, dental, and vision coverage

Unlimited PTO

H-1B sponsorship available

Job summary

Acceler8 Talent is seeking a Member of Technical Staff, Kernel Engineer, to design and optimize GPU kernels at the lowest layers of the AI inference stack. You will improve latency, throughput, memory efficiency, and hardware utilization across a distributed compute fleet in production environments.

The role is hands-on and ownership-focused, collaborating with inference, distributed systems, and infrastructure teams to push model performance on current and emerging accelerators.

Qualifications

• Strong experience in GPU programming, kernel development, high-performance computing, ML systems, or performance engineering.
• Proficiency in C++ and hands-on experience with CUDA, Triton, or another accelerator programming model.
• Familiarity with PyTorch and modern ML inference or training execution stacks.

Responsibilities

• Design, implement, and optimize high-performance GPU kernels for performance-critical AI operations, including GEMM, attention, normalization, quantization, KV-cache operations, and MoE workloads.
• Develop and optimize kernels using technologies such as CUDA, Triton, C++, PTX, CUTLASS, and related GPU programming frameworks.
• Profile production inference workloads to identify bottlenecks across compute, memory bandwidth, memory hierarchy, kernel launch overhead, synchronization, and data movement.
• Optimize GPU execution through techniques including memory coalescing, shared-memory utilization, tiling, warp-level programming, Tensor Core utilization, operator fusion, latency hiding, and compute/communication overlap.
• Improve end-to-end LLM inference performance across latency, throughput, memory utilization, concurrency, and hardware efficiency, rather than optimizing kernels in isolation.
• Develop and optimize kernels for modern model architectures, including Transformer-based LLMs, attention variants, MoE models, and emerging model architectures.
• Implement and evaluate lower-precision execution and quantization strategies, including FP16, BF16, FP8, FP4, INT8, and other hardware-supported formats.
• Integrate optimized kernels and operators into inference frameworks and internal runtimes built around technologies such as PyTorch, Triton, vLLM, SGLang, TensorRT-LLM, or equivalent systems.
• Use profiling and performance-analysis tools such as Nsight Systems, Nsight Compute, PyTorch Profiler, roofline analysis, and internal benchmarking infrastructure to diagnose and resolve performance regressions.
• Build reliable benchmarks, correctness tests, and performance regression tests to ensure kernel improvements remain numerically correct and production-ready.
• Optimize workloads across multi-GPU and distributed environments, working with the broader infrastructure team on communication, parallelism, and compute efficiency.
• Help extend the inference stack across heterogeneous hardware, including NVIDIA GPUs, AMD GPUs, and other current and emerging AI accelerators.
• Work closely with hardware vendors, inference engineers, and distributed-systems engineers to evaluate new accelerator architectures and translate hardware capabilities into production performance improvements.
• Contribute to architectural decisions affecting the company’s inference runtime, model execution layer, and hardware-performance roadmap.

Skills

GPU programming

Kernel development

High-performance computing

ML systems

Performance engineering

C++

CUDA

Triton

PyTorch

Tools

CUDA

Triton

CUTLASS

PyTorch

vLLM

SGLang

TensorRT-LLM

Job description

Acceler8 Talent is seeking a Member of Technical Staff, Kernel Engineer, to design and optimize GPU kernels at the lowest layers of the AI inference stack. You will improve latency, throughput, memory efficiency, and hardware utilization across a distributed compute fleet in production environments.

The role is hands-on and ownership-focused, collaborating with inference, distributed systems, and infrastructure teams to push model performance on current and emerging accelerators.