SDE, AI/ML Networking — Disaggregated Inference Expert

Amazon

Cupertino (CA)

On-site

USD 165,000 - 224,000

Full time

8 hours ago

Be an early applicant

Application generator

Stand out for this role — generate a tailored resume and cover letter in about a minute.

Get past ATS filters

Benefits offered by this job

Health insurance

RSUs

Paid time off

Job summary

Annapurna Labs, an integral part of AWS, is seeking a Software Development Engineer to push the frontier of AI/ML networking with disaggregated inference. You will build and optimize data-movement software across accelerators and servers, profiling workloads to approach hardware limits and collaborating across stacks from transport to inference frameworks.

You will join a team that designs hardware and software for EC2 infrastructure, focusing on ML/HPC workloads on AWS, with mentorship and

Qualifications

3+ years of non‑internship professional software development experience.
2+ years of non‑internship design or architecture experience.
Experience programming with at least one software language (C/C++ highlighted).
Experience with C/C++.

Responsibilities

Build and optimize the low-level data-movement software that transfers KV cache and activations across accelerators, servers, and heterogeneous memory — over AWS's highest-performance network fabric.
Profile real workloads, find the true bottleneck, and close the gap between "it works" and "it runs fast" — pushing components toward the hardware's limit.
Work across the stack — from network transport up to the inference frameworks — learning from the teams building the chips, runtime, and models.
Deliver features that ship to our largest clusters, for our largest customers, serving the largest AI models in production.