AI/ML Systems Engineer - High-Performance Inference

Amazon Inc.; Cupertino (CA); On-site; USD 165,000 - 224,000; Full time

Benefits offered by this job

Health insurance

401(k) matching

Paid time off

Parental leave

Job summary

Amazon's Annapurna Labs team is seeking a Software Development Engineer for AI/ML in AWS Neuron, focusing on model inference and performance tuning on Trainium and Inferentia silicon. You will work across the stack from frameworks to hardware to push the boundaries of AI acceleration.

You'll collaborate with compiler, runtime, and ML teams, optimize for latency and throughput, and engage with customers to enable their models on AWS accelerators, contributing to kernel design and scalable

Qualifications

• Bachelor's degree in computer science or equivalent.
• 3+ years of professional software development.
• Experience in ML model acceleration and inference optimization.
• Strong knowledge of system performance and memory management.
• Proficiency in C++ and Python.

Responsibilities

• Design, develop, and optimize machine learning models and frameworks for deployment on custom ML hardware accelerators.
• Participate in all stages of the ML system development lifecycle including distributed computing based architecture design, implementation, performance profiling, hardware-specific optimizations, testing and production deployment.
• Build infrastructure to systematically analyze and onboard multiple models with diverse architecture.
• Design and implement high-performance kernels and features for ML operations, leveraging the Neuron architecture and programming models.
• Analyze and optimize system-level performance across multiple generations of Neuron hardware.
• Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks.
• Implement optimizations such as fusion, sharding, tiling, and scheduling.
• Conduct comprehensive testing, including unit and end-to-end model testing with continuous deployment and releases through pipelines.
• Work directly with customers to enable and optimize their ML models on AWS accelerators.
• Collaborate across teams to develop innovative optimization techniques.

Skills

C++

Python

Performance profiling

Parallel computing

ML basics

Education

Bachelor's degree in Computer Science or equivalent

Tools

PyTorch

JAX

CUDA

TensorRT

Job description

Amazon's Annapurna Labs team is seeking a Software Development Engineer for AI/ML in AWS Neuron, focusing on model inference and performance tuning on Trainium and Inferentia silicon. You will work across the stack from frameworks to hardware to push the boundaries of AI acceleration.

You'll collaborate with compiler, runtime, and ML teams, optimize for latency and throughput, and engage with customers to enable their models on AWS accelerators, contributing to kernel design and scalable