RDMA-First GPU Networking & Distributed Inference Engineer

Baseten; New York (NY); On-site; USD 185,000 - 250,000; Full time

Benefits offered by this job

Competitive compensation

100% medical coverage

Generous PTO policy

Paid parental leave

401(k) contributions

Networking opportunities with ML startups

Job summary

A leading AI technology company is seeking a foundational engineer to lead GPU Networking efforts. This role involves architecting software for unifying GPUs, integrating high-performance networking capabilities, and optimizing large-scale AI models. The ideal candidate will possess deep experience with InfiniBand and RoCE, fluency in C++ or Python, and the ability to enhance system performance. Competitive compensation and benefits are offered, including equity and comprehensive insurance coverage.

Qualifications

• Deep experience with InfiniBand, RoCE v2 protocols.
• Fluency in C++ or Python, with knowledge of NVIDIA hardware.
• Experience with networking performance characterization.
• Ability to optimize data movement and system performance.

Responsibilities

• Integrate RDMA/RoCE capabilities into our inference stack.
• Tune networking layers for efficient communication.
• Enable fast startup speeds for large models.
• Design tools for visualizing packet flow and performance.
• Develop custom communication kernels.

Skills

High-performance networking protocols

C++ or Python

Memory hierarchy in NVIDIA architectures

Custom C++/Python bindings

Debugging NVLink topology

Tools

NCCL

NVSHMEM

UCX

GPUDirect Storage

Job description

A leading AI technology company is seeking a foundational engineer to lead GPU Networking efforts. This role involves architecting software for unifying GPUs, integrating high-performance networking capabilities, and optimizing large-scale AI models. The ideal candidate will possess deep experience with InfiniBand and RoCE, fluency in C++ or Python, and the ability to enhance system performance. Competitive compensation and benefits are offered, including equity and comprehensive insurance coverage.