Systems Engineer, High‑Speed AI Inference Networking

Amazon; Cupertino (CA); On-site; USD 165,000 - 224,000; Full time

Benefits offered by this job

Health insurance

RSUs / stock options

401(k) matching

Paid time off

Parental leave

Job summary

Annapurna Labs in Cupertino, CA, part of AWS, is seeking a Software Development Engineer focusing on AI/ML networking disaggregated inference. You will build low-level data movement software across accelerators and servers to minimize latency in LLM serving.

You will profile workloads, identify bottlenecks, and push components toward hardware limits while collaborating across chips, runtimes, and models teams. Prior AI/ML experience not required.

Qualifications

• Strong C/C++ and a genuine interest in low-level, performance-critical systems — solid command of Linux, memory, and writing fast code.
• Exposure to high-speed networking or HPC interconnects (RDMA, InfiniBand, libfabric, UCX, NCCL, MPI) is a strong plus; embedded-systems experience is welcome.
• Prior AI/ML experience is not required — if you're a strong systems engineer eager to learn, we'll teach you the ML side.

Responsibilities

• Build and optimize the low-level data‑movement software that transfers KV cache and activations across accelerators, servers, and heterogeneous memory — over AWS's highest-performance network fabric.
• Profile real workloads, find the true bottleneck, and close the gap between "it works" and "it runs fast" — pushing components toward the hardware's limit.
• Work across the stack — from network transport up to the inference frameworks — learning from the teams building the chips, runtime, and models.
• Deliver features that ship to our largest clusters, for our largest customers, serving the largest AI models in production.

Skills

C/C++

Linux

Networking concepts

Education

Bachelor's degree in computer science or equivalent

Job description

Annapurna Labs in Cupertino, CA, part of AWS, is seeking a Software Development Engineer focusing on AI/ML networking disaggregated inference. You will build low-level data movement software across accelerators and servers to minimize latency in LLM serving.

You will profile workloads, identify bottlenecks, and push components toward hardware limits while collaborating across chips, runtimes, and models teams. Prior AI/ML experience not required.