Staff AI Inference Performance Engineer

Gimlet Labs, Inc.; San Francisco (CA); On-site; USD 150,000 - 390,000; Full time

Job summary

Gimlet Labs, Inc. in San Francisco seeks a Member of Technical Staff to characterize and improve production AI inference across its heterogeneous accelerator fleet.

You will analyze kernel execution, compiler output, runtime behavior, memory movement, and scheduling to drive end-to-end latency and throughput improvements. You will build measurement systems and performance methodologies to identify bottlenecks, quantify cost drivers, and implement changes across kernels, runtimes, and distributed

Qualifications

• Bachelor's degree in a relevant field or equivalent experience.
• Experience profiling accelerator workloads and turning measurements into production improvements.
• Strong software engineering fundamentals and performance-critical systems.

Responsibilities

• Establish trustworthy performance baselines across accelerator architectures.
• Isolate bottlenecks across compute, memory bandwidth, cache behavior, kernel launch, synchronization, host-device interaction, communication, and runtime scheduling.
• Use profiling, hardware counters, controlled experiments, and performance models to explain workload behavior and identify levers to optimize.
• Improve execution through launch reduction, graph capture, stream and event coordination, compute-communication overlap, memory reuse, batching, and scheduling.
• Build automated benchmarks and regression detection to preserve performance across models, tensor shapes, data types, software releases, and hardware targets.
• Translate performance findings into production improvements, collaborating with kernel, compiler, ML systems, and distributed systems engineers when necessary.
• Develop a repeatable methodology for evaluating and bringing up new accelerators within Gimlet's execution platform.

Skills

Performance optimization

GPU profiling

Experiment design

Software engineering fundamentals

Education

Bachelor's degree or equivalent

Tools

Nsight Systems

Nsight Compute

rocprof

Omniperf

VTune

Perfetto

CUDA

HIP

Triton

Job description

Gimlet Labs, Inc. in San Francisco seeks a Member of Technical Staff to characterize and improve production AI inference across its heterogeneous accelerator fleet.

You will analyze kernel execution, compiler output, runtime behavior, memory movement, and scheduling to drive end-to-end latency and throughput improvements. You will build measurement systems and performance methodologies to identify bottlenecks, quantify cost drivers, and implement changes across kernels, runtimes, and distributed