Software Engineer: AI/ML Inference on AWS Neuron

Amazon Inc.; Cupertino (CA); On-site; USD 165,000 - 224,000; Full time

Benefits offered by this job

Health insurance

RSUs and sign-on payments

401(k) matching

Paid time off

Parental leave

Job summary

Amazon’s Annapurna Labs team at AWS Neuron delivers deep learning acceleration for Inferentia and Trainium. You’ll help architect distributed inference, tune models for latency and throughput, and collaborate across compiler, runtime, and hardware teams to maximize performance.

The role emphasizes Python and C++ development, system-level optimizations, and close work with customers to enable model deployment on AWS accelerators.

Qualifications

• Bachelor's degree in computer science or equivalent.
• 3+ years of non-internship professional software development experience.
• 3+ years of design/architecture of systems.

Responsibilities

• Design, develop, and optimize ML models and frameworks for deployment on custom ML hardware accelerators.

Skills

C++

Python

ML fundamentals

LLMs

Parallel computing

Debugging

System performance

Education

Bachelor's degree in computer science or equivalent

Tools

PyTorch

JIT compilation

AOT tracing

CUDA kernels

Job description

Amazon’s Annapurna Labs team at AWS Neuron delivers deep learning acceleration for Inferentia and Trainium. You’ll help architect distributed inference, tune models for latency and throughput, and collaborate across compiler, runtime, and hardware teams to maximize performance.

The role emphasizes Python and C++ development, system-level optimizations, and close work with customers to enable model deployment on AWS accelerators.