Senior AI/ML Systems Engineer, Neuron Inference

Amazon Inc.; Cupertino (CA); On-site; USD 193,000 - 262,000; Full time

Benefits offered by this job

Health insurance

401(k) matching

Paid time off

Parental leave

Job summary

Amazon’s Annapurna Labs team is seeking a Sr. Software Development Engineer for AI/ML on AWS Neuron to accelerate GenAI workloads on Inferentia/Trainium. You will architect features, mentor engineers, and optimize ML models and kernels for high-performance inference across clusters.

You will collaborate with cross-functional teams to push the boundaries of AI acceleration, participate in performance tuning, and work closely with customers to enable and optimize models on AWS accelerators.

Qualifications

• Bachelor's degree in computer science or equivalent.
• 5+ years of professional software development experience.
• 5+ years of programming in at least one language.
• 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience.
• Experience as a mentor, tech lead or leading an engineering team.
• Fundamentals of Machine learning and LLMs, their architecture, training and inference lifecycles along with work experience on some optimizations for improving the model execution.
• Software development experience in C++, Python (experience in at least one language is required).
• Strong understanding of system performance, memory management, and parallel computing principles.
• Deep understanding of computer architecture, operation systems level software and working knowledge of parallel computing.

Responsibilities

• Design, develop, and optimize machine learning models and frameworks for deployment on custom ML hardware accelerators.
• Participate in all stages of the ML system development lifecycle including distributed computing based architecture design, implementation, performance profiling, hardware-specific optimizations, testing and production deployment.
• Build infrastructure to systematically analyze and onboard multiple models with diverse architecture.
• Design and implement high-performance kernels and features for ML operations, leveraging the Neuron architecture and programming models
• Analyze and optimize system-level performance across multiple generations of Neuron hardware
• Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks
• Implement optimizations such as fusion, sharding, tiling, and scheduling
• Conduct comprehensive testing, including unit and end-to-end model testing with continuous deployment and releases through pipelines.
• Work directly with customers to enable and optimize their ML models on AWS accelerators
• Collaborate across teams to develop innovative optimization techniques

Skills

C++

Python

Debugging

Profiling

ML fundamentals

LLMs

Mentorship

Parallel computing

Education

Bachelor's degree in CS

Master's degree in CS

Tools

CUDA kernels

PyTorch

JIT compilation

TensorRT

Job description

Amazon’s Annapurna Labs team is seeking a Sr. Software Development Engineer for AI/ML on AWS Neuron to accelerate GenAI workloads on Inferentia/Trainium. You will architect features, mentor engineers, and optimize ML models and kernels for high-performance inference across clusters.

You will collaborate with cross-functional teams to push the boundaries of AI acceleration, participate in performance tuning, and work closely with customers to enable and optimize models on AWS accelerators.