Software Engineering Manager, ML Kernel Performance, AWS Neuron, Annapurna Labs

Amazon; Cupertino (CA); On-site; USD 212,700 - 287,700; Full time

Benefits offered by this job

Health insurance

401(k) matching

Parental leave

Paid time off

Mental Health Support

Job summary

Amazon Annapurna Labs in Cupertino seeks an Software Engineering Manager to lead ML kernel performance initiatives within AWS Neuron. You will mentor engineers and drive kernel optimizations, shaping compiler strategies and ML accelerator workloads.

Lead cross-functional efforts across frameworks, compilers, and runtime layers, partnering with customers to optimize their models on Inferentia and Trainium, while delivering scalable, high-impact solutions.

Qualifications

• 3+ years of engineering team management experience.
• 7+ years of experience working directly within engineering teams.
• 3+ years of designing or architecting new and existing systems, including design patterns, reliability and scaling.
• 8+ years of leading the definition and development of multi-tier web services.
• Knowledge of engineering practices and patterns across the full software/hardware/network development life cycle, including coding standards, code reviews, source control management, build processes, testing, certification, and livesite operations.
• Experience partnering with product or program management teams.

Responsibilities

• Design and implement high-performance compute kernels for ML operations, leveraging the Neuron architecture and programming models.
• Analyze and optimize kernel-level performance across multiple generations of Neuron hardware.
• Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks.
• Implement compiler optimizations such as fusion, sharding, tiling, and scheduling.
• Work directly with customers to enable and optimize their ML models on AWS accelerators.
• Collaborate across teams to develop innovative kernel optimization techniques.
• Build high‑impact solutions, create metrics, implement automation, and resolve root causes of software defects.
• Participate in design discussions, code review, and communicate with internal and external stakeholders.

Skills

Engineering team management

Hands-on engineering

System design & architecture

Multi-tier web services

Product/Program management partnership

Technical leadership

Job description

Software Engineering Manager, ML Kernel Performance, AWS Neuron, Annapurna Labs

The Annapurna Labs team at Amazon Web Services (AWS) builds AWS Neuron, the software development kit used to accelerate deep learning and GenAI workloads on Amazon’s custom machine learning accelerators, Inferentia and Trainium. The Acceleration Kernel Library team focuses on maximizing performance for AWS's custom ML accelerators, crafting high-performance kernels for ML functions to deliver optimal performance for demanding workloads. As part of the Neuron Compiler organization, the team works across multiple technology layers—from frameworks and compilers to runtime and collectives—optimizing current performance and contributing to future architecture designs while collaborating with customers to enable their models.

Key Responsibilities

• Design and implement high-performance compute kernels for ML operations, leveraging the Neuron architecture and programming models.
• Analyze and optimize kernel-level performance across multiple generations of Neuron hardware.
• Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks.
• Implement compiler optimizations such as fusion, sharding, tiling, and scheduling.
• Work directly with customers to enable and optimize their ML models on AWS accelerators.
• Collaborate across teams to develop innovative kernel optimization techniques.
• Build high‑impact solutions, create metrics, implement automation, and resolve root causes of software defects.
• Participate in design discussions, code review, and communicate with internal and external stakeholders.

A Day in the Life

In a startup‑like environment, you will design and code solutions that drive efficiencies in software architecture, create metrics, implement automation, and resolve the root cause of software defects. You will collaborate cross‑functionally to influence business decisions with your technical input.

Basic Qualifications

• 3+ years of engineering team management experience.
• 7+ years of experience working directly within engineering teams.
• 3+ years of designing or architecting new and existing systems, including design patterns, reliability and scaling.
• 8+ years of leading the definition and development of multi‑tier web services.
• Knowledge of engineering practices and patterns across the full software/hardware/network development life cycle, including coding standards, code reviews, source control management, build processes, testing, certification, and livesite operations.
• Experience partnering with product or program management teams.

Preferred Qualifications

• Experience communicating with users, other technical teams, and senior leadership to collect requirements, describe software product features, technical designs, and product strategy.
• Experience recruiting, hiring, mentoring/coaching, and managing teams of software engineers to improve their skills and effectiveness.

Benefits

Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance, and optional Supplemental life plans), EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage, 401(k) matching, paid time off, and parental leave. Learn more at https://amazon.jobs/en/benefits.

Salary

USA, CA, Cupertino: $212,700.00 – $287,700.00 USD annually.

Equal Opportunity Employer

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Posted: July 16, 2026