AI Performance & Kernel Optimization Engineer

Bison; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Medical benefits

Dental & Vision

FSA plans

401(k)

Relocation support

In-office meals

Unlimited PTO

Team in SF

Job summary

Zyphra, an AI company based in San Francisco, seeks a Research Engineer to advance AI performance and kernel optimization for large-scale language model training and inference. You will work with pretraining and inference teams to identify bottlenecks and implement highly optimized kernels, pushing throughput and latency improvements across accelerator platforms.

You will tackle memory, scheduling, and compute utilization bottlenecks, optimize distributed systems, and collaborate closely with

Qualifications

• Excellent low-level performance intuition and the ability to reason about hardware-software interactions.
• Experience writing highly performant GPU kernels at any level of abstraction (PTX, CUDA, HIP, Triton, or similar).
• Experience with non-NVIDIA accelerator hardware (AMD, AWS Trainium, Google TPU, Qualcomm, ARM, Intel, etc.).
• Strong understanding of distributed training systems and parallelism (data/model/pipeline), sharding, and overlap between computation and communication.

Responsibilities

• Kernel development and optimization for large-scale ML workloads across CUDA, HIP, Triton, or other GPU DSLs.
• Performance tuning for training and inference on GPUs and accelerators.
• Profiling and eliminating bottlenecks in memory movement, communication, scheduling, and compute utilization.
• Optimizing distributed training and inference systems for large MoE models with model parallelism.
• Portability and optimization across non-NVIDIA hardware, including AMD MI-series.”
• Collaborate with research and infrastructure teams to translate systems improvements into real-world model gains.

Skills

Low-level performance intuition

GPU kernel programming

Distributed training

Profiling & debugging

Hardware awareness

Communication & collaboration

Tools

CUDA

HIP

Triton

PTX

Job description

Zyphra, an AI company based in San Francisco, seeks a Research Engineer to advance AI performance and kernel optimization for large-scale language model training and inference. You will work with pretraining and inference teams to identify bottlenecks and implement highly optimized kernels, pushing throughput and latency improvements across accelerator platforms.

You will tackle memory, scheduling, and compute utilization bottlenecks, optimize distributed systems, and collaborate closely with