Senior Engineering Manager — ML Kernel Performance

Amazon; Toronto; On-site; CAD 171,000 - 286,000; Full time

Job summary

Amazon's Annapurna Labs team within AWS Neuron seeks a Software Engineering Manager to lead ML kernel performance initiatives for Inferentia and Trainium. You will architect high-performance kernels, guide a feature roadmap, and mentor engineers across compiler, runtime, and hardware domains to optimize ML workloads.

You will collaborate with customers and partner teams, analyze bottlenecks with profiling tools, and drive design discussions from concept to production.

Qualifications

• 3+ years of engineering team management experience.
• 7+ years of working directly within engineering teams experience.
• 3+ years of designing or architecting (design patterns, reliability and scaling) of new and existing systems experience.
• 8+ years of leading the definition and development of multi tier web services experience.
• Knowledge of engineering practices and patterns for the full software/hardware/networks life cycle, including coding standards, code reviews, source control, build, testing, and live-site operations.
• Experience partnering with product or program management teams.

Responsibilities

• Design and implement high-performance compute kernels for ML operations, leveraging the Neuron architecture and programming models.
• Analyze and optimize kernel-level performance across multiple generations of Neuron hardware.
• Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks.
• Implement compiler optimizations such as fusion, sharding, tiling, and scheduling.
• Work directly with customers to enable and optimize their ML models on AWS accelerators.
• Collaborate across teams to develop innovative kernel optimization techniques.

Skills

Engineering management

Team leadership

Engineering experience

Web services

PM collaboration

Job description

Amazon's Annapurna Labs team within AWS Neuron seeks a Software Engineering Manager to lead ML kernel performance initiatives for Inferentia and Trainium. You will architect high-performance kernels, guide a feature roadmap, and mentor engineers across compiler, runtime, and hardware domains to optimize ML workloads.

You will collaborate with customers and partner teams, analyze bottlenecks with profiling tools, and drive design discussions from concept to production.