GPU Networking Engineer - RDMA & Distributed Inference

Baseten; San Francisco (CA); On-site; Full time

Benefits offered by this job

Competitive compensation

100% medical, dental, and vision insurance

Generous PTO policy

Paid parental leave

401(k)

Exposure to ML startups

Job summary

A cutting-edge AI infrastructure company in San Francisco seeks an experienced network engineer to optimize high-performance networking protocols for AI models. The ideal candidate will integrate RDMA and InfiniBand into the inference stack, ensuring efficient communication and low latency. A deep understanding of NVIDIA architectures and experience with C++ or Python are essential. This role offers competitive compensation and substantial equity, alongside comprehensive benefits including health insurance and generous PTO.

Qualifications

• Deep experience with InfiniBand and RoCE v2.
• Ability to bridge high-level logic with hardware.
• Experience diving into TensorRT-LLM source code.

Responsibilities

• Integrate RDMA/RoCE/InfiniBand capabilities into inference stack.
• Implement networking layers for efficient Disaggregated KV Cache Offload.
• Design tools for visualizing packet flow and diagnosing system behaviors.

Skills

High-performance networking protocols

C++ or Python

Memory hierarchy in NVIDIA architectures

Debugging NVLink topology

Tools

NCCL

NVSHMEM

UCX

Job description

A cutting-edge AI infrastructure company in San Francisco seeks an experienced network engineer to optimize high-performance networking protocols for AI models. The ideal candidate will integrate RDMA and InfiniBand into the inference stack, ensuring efficient communication and low latency. A deep understanding of NVIDIA architectures and experience with C++ or Python are essential. This role offers competitive compensation and substantial equity, alongside comprehensive benefits including health insurance and generous PTO.