AI/ML Systems Engineer - Neuron Inference

Amazon; Seattle (WA); On-site; USD 144,000 - 194,000; Full time

Job summary

Amazon's Annapurna Labs team is seeking a senior software engineer to design and optimize ML models and frameworks for Neuron, AWS's ML accelerators. You will work across PyTorch/JAX stacks, implement high-performance kernels, and collaborate with customers to tune inference and training workloads.

You will lead performance profiling, system-level optimizations, and contribute to future architecture. A startup-like, mentorship-rich culture with hands-on coding and cross-team collaboration awaits.

Qualifications

• 3+ years of non-internship professional software development experience.
• Bachelor's degree in computer science or equivalent.
• 3+ years of design or architecture experience in large systems.
• Fundamentals of ML and LLMs, including model lifecycles and optimizations.
• Proficiency in C++ and Python; strong debugging and profiling skills.

Responsibilities

• Design, develop, and optimize ML models and frameworks for custom ML hardware accelerators.
• Participate in all stages of ML system development including distributed architecture design and production deployment.
• Build infrastructure to analyze and onboard multiple models with diverse architecture.
• Design and implement high-performance kernels for ML operations using Neuron.
• Analyze and optimize system-level performance across Neuron hardware generations.
• Conduct detailed performance analysis to identify and resolve bottlenecks.
• Implement optimizations such as fusion, sharding, tiling, and scheduling.
• Collaborate with customers to enable and optimize ML models on AWS accelerators.

Skills

C++

Python

Machine learning basics

LLMs

Performance optimization

Debugging & profiling

Distributed systems

Education

Bachelor's degree in computer science or equivalent

Tools

PyTorch

JIT compilation

AOT tracing

CUDA kernels

TensorRT

Job description

Amazon's Annapurna Labs team is seeking a senior software engineer to design and optimize ML models and frameworks for Neuron, AWS's ML accelerators. You will work across PyTorch/JAX stacks, implement high-performance kernels, and collaborate with customers to tune inference and training workloads.

You will lead performance profiling, system-level optimizations, and contribute to future architecture. A startup-like, mentorship-rich culture with hands-on coding and cross-team collaboration awaits.