Software Engineer - AI/ML Disaggregated Inference

Amazon Web Services (AWS); Cupertino (CA); On-site; USD 165,000 - 224,000; Full time

Job summary

Annapurna Labs, part of AWS, seeks a systems engineer to design and optimize high-speed data-m movement for disaggregated inference across accelerators, servers, and heterogeneous memory. You will push toward hardware limits and deliver fast, scalable components for AI workloads on AWS.

Experience with C/C++, Linux, and performance-critical systems is essential; prior AI/ML experience is not required, as we will teach the ML aspects.

Qualifications

• 3+ years of non-internship professional software development experience.
• 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience.
• Experience programming with at least one software programming language.
• Experience with C/C++.

Responsibilities

• Build and optimize the low-level data-movement software that transfers KV cache and activations across accelerators, servers, and heterogeneous memory — over AWS's highest-performance network fabric.
• Profile real workloads, find the true bottleneck, and close the gap between "it works" and "it runs fast" — pushing components toward the hardware's limit.
• Work across the stack — from network transport up to the inference frameworks — learning from the teams building the chips, runtime, and models.
• Deliver features that ship to our largest clusters, for our largest customers, serving the largest AI models in production.

Skills

C/C++

Education

Bachelor's degree in computer science or equivalent

Job description

Annapurna Labs, part of AWS, seeks a systems engineer to design and optimize high-speed data-m movement for disaggregated inference across accelerators, servers, and heterogeneous memory. You will push toward hardware limits and deliver fast, scalable components for AI workloads on AWS.

Experience with C/C++, Linux, and performance-critical systems is essential; prior AI/ML experience is not required, as we will teach the ML aspects.