Senior AI/ML Systems Engineer - Disaggregated Inference

Energy Jobline ZR; Cupertino (CA); On-site; USD 193,000 - 262,000; Full time

Benefits offered by this job

Health insurance

401(k) matching

Paid time off

Parental leave

Job summary

Annapurna Labs, an integral part of AWS, is seeking a systems-focused engineer to advance disaggregated inference for large AI models. You will work on high-speed data movement across accelerators and memory, optimizing the path that powers real-time ML at scale.

In this role, you will collaborate across kernel, network, runtime, and models teams, mentoring others and delivering features for the world's largest clusters.

Qualifications

• 5+ years of non professional software development experience
• 5+ years of programming with at least one software programming experience
• 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience
• 5+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
• Experience as a mentor, tech lead or leading an engineering team
• Must have C/C++ Coding Experience

Responsibilities

• Build and optimize the low-level data-movement software across accelerators, servers, and heterogeneous memory — over AWS's highest-performance network fabric.
• Push performance to the hardware roofline: profile, find the real bottleneck, and close the gap between "it works" and "it runs as fast as physics permits."
• Work across the stack — from kernel and network transport up to the inference frameworks — and partner with teams building the chips, runtime, and models.
• Deliver features that run on our largest clusters, for our largest customers, serving the largest AI models in production.

Skills

C/C++ coding

Linux kernel

Performance profiling

High-speed networking

Tech leadership

Education

Bachelor's degree in computer science or equivalent

Tools

NCCL

NIXL

NVSHMEM

Job description

Annapurna Labs, an integral part of AWS, is seeking a systems-focused engineer to advance disaggregated inference for large AI models. You will work on high-speed data movement across accelerators and memory, optimizing the path that powers real-time ML at scale.

In this role, you will collaborate across kernel, network, runtime, and models teams, mentoring others and delivering features for the world's largest clusters.