Senior AI/ML Systems Engineer - Flexible Hours & Networking

Amazon; Cupertino (CA); On-site; USD 193,000 - 262,000; Full time

Job summary

Amazon's Annapurna Labs in Cupertino seeks a Sr. Software Development Engineer to push the limits of disaggregated inference for AI/ML workloads. You will help split LLM serving across prefills and decode pools, moving KV caches at hardware limits to minimize latency.

You will own low-level data movement across accelerators, servers, and memory, and collaborate with chip, runtime, and model teams to deliver scalable features for AWS EC2-scale deployments.

Qualifications

• 5+ years of non-internship professional software development experience.
• 5+ years of programming with at least one software programming language experience.
• 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience.
• 5+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience.
• Experience as a mentor, tech lead or leading an engineering team.
• Must have C/C++ Coding Experience

Responsibilities

• Build and optimize the low-level data-movement software across accelerators, servers, and heterogeneous memory — over AWS's highest-performance network fabric.
• Push performance to the hardware roofline: profile, find the real bottleneck, and close the gap between 'it works' and 'it runs as fast as physics permits.'
• Work across the stack — from kernel and network transport up to the inference frameworks — and partner with teams building the chips, runtime, and models.
• Deliver features that run on our largest clusters, for our largest customers, serving the largest AI models in production.

Skills

C/C++

Low-level programming

Performance optimization

Mentoring/leadership

Linux/kernel knowledge

Education

Bachelor's degree in computer science or equivalent

Tools

RDMA/InfiniBand

libfabric/UCX

MPI

Job description

Amazon's Annapurna Labs in Cupertino seeks a Sr. Software Development Engineer to push the limits of disaggregated inference for AI/ML workloads. You will help split LLM serving across prefills and decode pools, moving KV caches at hardware limits to minimize latency.

You will own low-level data movement across accelerators, servers, and memory, and collaborate with chip, runtime, and model teams to deliver scalable features for AWS EC2-scale deployments.