AI/ML Inference Engineer for AWS Neuron

Amazon; Cupertino (CA); On-site; USD 165,000 - 224,000; Full time

Benefits offered by this job

Health insurance

401(k) matching

Paid time off

Parental leave

RSUs

Job summary

Amazon's Annapurna Labs team at AWS is seeking a Software Development Engineer focused on AI/ML for AWS Neuron, the runtime and compiler stack that accelerates models on Inferentia and Trainium. You will design high-performance kernels, optimize inference across large models, and collaborate with scientists, engineers, and customers to push the boundaries of AI acceleration.

This role offers a startup-like culture with mentorship, code reviews, and opportunities to impact state-of-the-art

Qualifications

• Bachelor's degree in computer science or equivalent.
• 3+ years of non-internship professional software development experience.
• 3+ years of non-internship design or architecture experience.
• Fundamentals of Machine learning and LLMs, their architecture, training and inference lifecycles with optimization experience.
• Software development in C++, Python (at least one language required).
• Strong understanding of system performance, memory management, and parallel computing principles.
• Proficiency in debugging and scalable software engineering practices.

Responsibilities

• Design, develop, and optimize ML models and frameworks for deployment on custom ML hardware accelerators.
• Participate in all stages of ML system development lifecycle including distributed architecture design, performance profiling, and production deployment.
• Build infrastructure to analyze and onboard multiple models with diverse architectures.
• Design and implement high-performance kernels and features for ML operations leveraging Neuron architecture.
• Analyze and optimize system-level performance across Neuron hardware generations.
• Conduct detailed performance analysis using profiling tools to identify bottlenecks.
• Implement optimizations such as fusion, sharding, tiling, and scheduling.
• Conduct comprehensive testing including unit and end-to-end model testing with CI/CD pipelines.
• Work directly with customers to enable and optimize their ML models on AWS accelerators.
• Collaborate across teams to develop innovative optimization techniques.

Skills

C++

Python

System performance

Debugging

Parallel computing

Education

Bachelor's degree in computer science

Tools

PyTorch

JAX

CUDA kernels

TensorRT

CUTLASS

Job description

Amazon's Annapurna Labs team at AWS is seeking a Software Development Engineer focused on AI/ML for AWS Neuron, the runtime and compiler stack that accelerates models on Inferentia and Trainium. You will design high-performance kernels, optimize inference across large models, and collaborate with scientists, engineers, and customers to push the boundaries of AI acceleration.

This role offers a startup-like culture with mentorship, code reviews, and opportunities to impact state-of-the-art