Senior Systems Engineer, AI Inference & HPC Networking

Amazon; Cupertino (CA); On-site; USD 193,000 - 262,000; Full time

Benefits offered by this job

RSUs

Health insurance

401(k) matching

Paid time off

Job summary

Annapurna Labs, a crucial part of AWS, seeks a systems engineer to advance disaggregated inference for EC2. You will optimize data movement across accelerators, servers, and memory to enable ultra-low latency for the world’s largest models.

Join a fast-moving, globally distributed team; work with HPC interconnects, kernel networking, and ML frameworks, mentoring others and driving performance to the hardware roofline. This role offers mentorship and growth in a dynamic AWS environment.

Qualifications

• 5+ years of non-internship professional software development experience.
• 5+ years of programming with at least one software programming language experience.
• 5+ years of leading design or architecture of new and existing systems.
• 5+ years of full software development lifecycle including coding standards, reviews, source control, build processes, testing, and operations.
• Experience as a mentor, tech lead or leading an engineering team.
• Must have C/C++ coding experience.

Responsibilities

• Build and optimize the low‑level data-movement software across accelerators, servers, and heterogeneous memory — over AWS's highest-performance network fabric.
• Push performance to the hardware roofline: profile, bottleneck analysis, and close the gap between 'it works' and 'it runs as fast as physics permits'.
• Work across the stack — from kernel and network transport up to the inference frameworks — and partner with teams building the chips, runtime, and models.
• Deliver features that run on our largest clusters, for our largest customers, serving the largest AI models in production.

Skills

C/C++

Low-level programming

Linux

Performance optimization

Networking

Mentor / tech lead

HPC interconnects

ML experience

Education

Bachelor's degree in computer science or equivalent

Tools

NCCL

NIXL

NVSHMEM

Job description

Annapurna Labs, a crucial part of AWS, seeks a systems engineer to advance disaggregated inference for EC2. You will optimize data movement across accelerators, servers, and memory to enable ultra-low latency for the world’s largest models.

Join a fast-moving, globally distributed team; work with HPC interconnects, kernel networking, and ML frameworks, mentoring others and driving performance to the hardware roofline. This role offers mentorship and growth in a dynamic AWS environment.