ML Systems Engineer: Inference & GPU-Driven Distributed Workloads

Bake AI; San Mateo, Northern (CA, KY); Hybrid; USD 180,000 - 240,000; Full time

Job summary

Bake AI in San Mateo, California, is hiring a Member of Technical Staff to build ML systems that enable sustained AI research. You will own performance-critical components across model serving, GPU execution, and distributed workloads, and you will push beyond a simple inference engine to improve correctness and efficiency.

You will work with researchers and infrastructure engineers, design robust systems, measure performance, and drive implementation from architecture through production, while

Qualifications

• Independently designed and shipped a substantial ML systems component in production or a research setting.
• Strong Python and systems programming with C++ or Rust and ability to modify an execution engine.
• Hands-on GPU programming and profiling with CUDA or Triton and memory hierarchy understanding.
• Experience with transformer execution, KV caches, and distributed LLM serving across multiple GPUs.

Responsibilities

• Design and improve inference infrastructure for long-running agents and research workloads.
• Extend or build inference runtimes when existing systems do not meet workload needs.
• Evaluate techniques for latency, throughput, memory, and cost and implement trade-offs.
• Profile full execution path from Python to GPU kernels and memory movement.
• Build distributed execution paths for training, post-training, and agent rollouts.
• Establish reproducible benchmarks and regression checks for correctness and performance.
• Own designs and production follow-through with strong code review and collaboration.

Skills

Python

C++

Rust

GPU programming

CUDA

Triton

Distributed systems

Performance optimization

Tools

vLLM

SGLang

Profiling tools

Job description

Bake AI in San Mateo, California, is hiring a Member of Technical Staff to build ML systems that enable sustained AI research. You will own performance-critical components across model serving, GPU execution, and distributed workloads, and you will push beyond a simple inference engine to improve correctness and efficiency.

You will work with researchers and infrastructure engineers, design robust systems, measure performance, and drive implementation from architecture through production, while