AI/ML Software Engineer: Neuron Inference & Acceleration

Amazon Inc.; Seattle (WA); On-site; USD 165,000 - 224,000; Full time

Job summary

Amazon Inc. in Cupertino, CA is seeking a Software Development Engineer for AI/ML in AWS Neuron, Model Inference.

You will design, implement, and optimize distributed ML inference software for Inferentia and Trainium accelerators, collaborating across compiler, runtime, framework, and hardware teams. The role requires deep knowledge of C++ and Python, strong system performance skills, and hands-on experience with ML frameworks like PyTorch.

Qualifications

• Bachelor's degree in computer science or equivalent.
• 3+ years of non-internship professional software development experience.
• Fundamentals of Machine learning and LLMs, their architecture, training and inference lifecycles.

Responsibilities

• Design, develop, and optimize machine learning models and frameworks for deployment on custom ML hardware accelerators.
• Participate in all stages of the ML system development lifecycle including distributed computing based architecture design, implementation, performance profiling, hardware-specific optimizations, testing and production deployment.
• Build infrastructure to systematically analyze and onboard multiple models with diverse architecture.
• Design and implement high-performance kernels and features for ML operations, leveraging the Neuron architecture and programming models.
• Analyze and optimize system-level performance across multiple generations of Neuron hardware.
• Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks.
• Implement optimizations such as fusion, sharding, tiling, and scheduling.
• Conduct comprehensive testing, including unit and end-to-end model testing with continuous deployment and releases through pipelines.
• Work directly with customers to enable and optimize their ML models on AWS accelerators.
• Collaborate across teams to develop innovative optimization techniques.

Skills

Python

C++

Machine learning

System performance

Debugging

Parallel computing

Education

Bachelor's degree in computer science or equivalent

Master's degree or Ph.D. in Computer Science or equivalent

Tools

PyTorch

JIT compilation

AOT tracing

CUDA kernels

TensorRT

Job description

Amazon Inc. in Cupertino, CA is seeking a Software Development Engineer for AI/ML in AWS Neuron, Model Inference.

You will design, implement, and optimize distributed ML inference software for Inferentia and Trainium accelerators, collaborating across compiler, runtime, framework, and hardware teams. The role requires deep knowledge of C++ and Python, strong system performance skills, and hands-on experience with ML frameworks like PyTorch.