Member of Technical Staff - GPU Performance

Gimlet Labs, Inc.; San Francisco (CA); On-site; USD 150,000 - 390,000; Full time

Job summary

Gimlet Labs, Inc. in San Francisco seeks a Member of Technical Staff to characterize and improve production AI inference across its heterogeneous accelerator fleet.

You will analyze kernel execution, compiler output, runtime behavior, memory movement, and scheduling to drive end-to-end latency and throughput improvements. You will build measurement systems and performance methodologies to identify bottlenecks, quantify cost drivers, and implement changes across kernels, runtimes, and distributed

Qualifications

• Bachelor's degree in a relevant field or equivalent experience.
• Experience profiling accelerator workloads and turning measurements into production improvements.
• Strong software engineering fundamentals and performance-critical systems.

Responsibilities

• Establish trustworthy performance baselines across accelerator architectures.
• Isolate bottlenecks across compute, memory bandwidth, cache behavior, kernel launch, synchronization, host-device interaction, communication, and runtime scheduling.
• Use profiling, hardware counters, controlled experiments, and performance models to explain workload behavior and identify levers to optimize.
• Improve execution through launch reduction, graph capture, stream and event coordination, compute-communication overlap, memory reuse, batching, and scheduling.
• Build automated benchmarks and regression detection to preserve performance across models, tensor shapes, data types, software releases, and hardware targets.
• Translate performance findings into production improvements, collaborating with kernel, compiler, ML systems, and distributed systems engineers when necessary.
• Develop a repeatable methodology for evaluating and bringing up new accelerators within Gimlet's execution platform.

Skills

Performance optimization

GPU profiling

Experiment design

Software engineering fundamentals

Education

Bachelor's degree or equivalent

Tools

Nsight Systems

Nsight Compute

rocprof

Omniperf

VTune

Perfetto

CUDA

HIP

Triton

Job description

About Us

Gimlet is building the first multi-silicon neocloud designed for fast, efficient AI inference.

About Us

Gimlet is building the first multi-silicon neocloud designed for fast, efficient AI inference. We combine large-scale compute infrastructure with an execution platform that partitions AI workloads and maps each stage to the hardware best suited to run it. We work with foundation labs, hyperscalers, and AI-native companies, giving our team access to technical problems spanning frontier models, production infrastructure, and emerging hardware.

About The Role

As a Member of Technical Staff, you will characterize and improve the performance of production AI inference across Gimlet's heterogeneous accelerator fleet. You will work across kernel execution, compiler output, runtime behavior, memory movement, synchronization, scheduling, and multi-device execution. Rather than treating performance as a collection of isolated microbenchmarks, you will connect hardware-level evidence to end-to-end latency, throughput, utilization, and cost. You will build the measurement systems and performance methodology that let Gimlet understand new hardware quickly, identify the limiting resource in real workloads, and turn that understanding into changes across kernels, compilers, runtimes, and distributed execution.

What Success Looks Like

• Establish trustworthy performance baselines for production inference workloads across established and emerging accelerator architectures
• Isolate bottlenecks across compute, memory bandwidth, cache behavior, kernel launch, synchronization, host-device interaction, communication, and runtime scheduling
• Use profiling, hardware counters, controlled experiments, and performance models to explain why a workload behaves as it does and identify the highest-leverage optimization
• Improve execution through changes such as launch reduction, graph capture, stream and event coordination, compute-communication overlap, memory reuse, batching, and scheduling
• Build automated benchmarks and regression detection that preserve performance across models, tensor shapes, data types, software releases, and hardware targets
• Translate performance findings into production improvements, partnering with kernel, compiler, ML systems, and distributed systems engineers when necessary
• Develop a repeatable methodology for evaluating and bringing up new accelerators within Gimlet's execution platform

You may be a good fit if you have

• Strong software engineering fundamentals and experience working on performance-critical systems
• Experience profiling GPU or accelerator workloads and turning measurements into production improvements
• A strong understanding of GPU execution, memory hierarchies, asynchronous execution, synchronization, and performance tradeoffs
• The ability to design controlled experiments, distinguish symptoms from root causes, and communicate performance findings precisely
• Bachelor's degree in a relevant field, or an equivalent combination of education, training, and professional experience

Strong candidates may also have

• Experience with Nsight Systems, Nsight Compute, rocprof, Omniperf, VTune, Perfetto, or hardware performance counters
• Experience with CUDA, HIP, Triton, or another accelerator programming model and the ability to prototype low-level optimizations
• Familiarity with roofline analysis, queueing behavior, occupancy, latency hiding, instruction throughput, and memory-bandwidth analysis
• Experience optimizing inference runtimes, execution graphs, memory allocators, kernel dispatch, batching, or model-serving systems
• Familiarity with collective communication, topology-aware execution, and compute-communication overlap across multiple accelerators
• Experience building performance laboratories, benchmark suites, regression infrastructure, or hardware bring-up methodology

Why join now?

• Solve hard problems.
• Own meaningful work.
• Build for production.
• Help define what’s next.

Compensation Range: $150K - $390K