Low-Latency Inference Systems Engineer (Multi-GPU)

Digital Waffle; San Francisco (CA); Hybrid; USD 180,000 - 280,000; Full time

Job summary

Digital Waffle in San Francisco is seeking a systems-focused engineer to own and optimize large-scale model serving. You will work on a fork of vLLM across an H100 fleet, focusing on latency and cost per token, with equity as part of the compensation.

The role involves kernel-level optimization, advanced batching strategies, and careful quantisation decisions to meet customer latency targets. A hybrid work setup is offered, and growth toward leading a larger inference team is expected.

Qualifications

• Backgrounds that translate: HPC, graphics, embedded, compiler work, or performance engineering where latency was a product requirement.

Responsibilities

• Kernel-level optimization using CUDA and Triton, custom ops, fused attention.
• Implement continuous batching, KV-cache strategy, and speculative decoding tailored per workload.
• Evaluate quantisation trade-offs where customers notice quality loss first.
• Perform multi-GPU and multi-node serving with profiling to prove improvements.

Skills

Latency optimization

Performance engineering

Multi-GPU scaling

Tools

CUDA

Triton

custom ops

Job description

Digital Waffle in San Francisco is seeking a systems-focused engineer to own and optimize large-scale model serving. You will work on a fork of vLLM across an H100 fleet, focusing on latency and cost per token, with equity as part of the compensation.

The role involves kernel-level optimization, advanced batching strategies, and careful quantisation decisions to meet customer latency targets. A hybrid work setup is offered, and growth toward leading a larger inference team is expected.