AI/ML Inference Engineer for AWS Neuron

Amazon Web Services (AWS); Cupertino (CA); On-site; USD 165,000 - 224,000; Full time

Job summary

Annapurna Labs (U.S.) Inc. in Cupertino, California, is seeking an experienced software engineer to design and optimize ML models and frameworks for deployment on custom hardware accelerators.

You will work across ML system lifecycle, implement high-performance kernels, optimize system-wide bottlenecks, and collaborate with customers to enable model acceleration on Neuron and Trainium/Inferentia. Strong fundamentals in C++, Python, ML, and distributed systems are required.

Qualifications

• Bachelor's degree in computer science or equivalent.
• 3+ years of professional software development experience.
• Experience in design or architecture of scalable systems.
• Fundamentals of ML, LLMs, and inference lifecycles.

Responsibilities

• Design, develop, and optimize ML models and frameworks for deployment on custom ML hardware accelerators.
• Participate in the ML system development lifecycle including distributed computing based architecture design, implementation, performance profiling, hardware-specific optimizations, testing and production deployment.
• Build infrastructure to analyze and onboard multiple models with diverse architecture.
• Design and implement high-performance kernels and features for ML operations, leveraging the Neuron architecture and programming models.
• Analyze and optimize system-level performance across multiple generations of Neuron hardware.
• Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks.
• Implement optimizations such as fusion, sharding, tiling, and scheduling.
• Conduct comprehensive testing, including unit and end-to-end model testing with pipelines.

Skills

Fundamentals of ML & LLMs

Performance optimization

Profiling & debugging

Parallel computing

Large-scale distributed systems

Education

Bachelor's degree in computer science or equivalent

Tools

C++

Python

Job description

Annapurna Labs (U.S.) Inc. in Cupertino, California, is seeking an experienced software engineer to design and optimize ML models and frameworks for deployment on custom hardware accelerators.

You will work across ML system lifecycle, implement high-performance kernels, optimize system-wide bottlenecks, and collaborate with customers to enable model acceleration on Neuron and Trainium/Inferentia. Strong fundamentals in C++, Python, ML, and distributed systems are required.