Engineering Manager, ML Kernel Performance

Amazon; Cupertino (CA); On-site; USD 212,700 - 287,700; Full time

Benefits offered by this job

Health insurance

401(k) matching

Parental leave

Paid time off

Mental Health Support

Job summary

Amazon Annapurna Labs in Cupertino seeks an Software Engineering Manager to lead ML kernel performance initiatives within AWS Neuron. You will mentor engineers and drive kernel optimizations, shaping compiler strategies and ML accelerator workloads.

Lead cross-functional efforts across frameworks, compilers, and runtime layers, partnering with customers to optimize their models on Inferentia and Trainium, while delivering scalable, high-impact solutions.

Qualifications

• 3+ years of engineering team management experience.
• 7+ years of experience working directly within engineering teams.
• 3+ years of designing or architecting new and existing systems, including design patterns, reliability and scaling.
• 8+ years of leading the definition and development of multi-tier web services.
• Knowledge of engineering practices and patterns across the full software/hardware/network development life cycle, including coding standards, code reviews, source control management, build processes, testing, certification, and livesite operations.
• Experience partnering with product or program management teams.

Responsibilities

• Design and implement high-performance compute kernels for ML operations, leveraging the Neuron architecture and programming models.
• Analyze and optimize kernel-level performance across multiple generations of Neuron hardware.
• Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks.
• Implement compiler optimizations such as fusion, sharding, tiling, and scheduling.
• Work directly with customers to enable and optimize their ML models on AWS accelerators.
• Collaborate across teams to develop innovative kernel optimization techniques.
• Build high‑impact solutions, create metrics, implement automation, and resolve root causes of software defects.
• Participate in design discussions, code review, and communicate with internal and external stakeholders.

Skills

Engineering team management

Hands-on engineering

System design & architecture

Multi-tier web services

Product/Program management partnership

Technical leadership

Job description

Amazon Annapurna Labs in Cupertino seeks an Software Engineering Manager to lead ML kernel performance initiatives within AWS Neuron. You will mentor engineers and drive kernel optimizations, shaping compiler strategies and ML accelerator workloads.

Lead cross-functional efforts across frameworks, compilers, and runtime layers, partnering with customers to optimize their models on Inferentia and Trainium, while delivering scalable, high-impact solutions.