Sr. Software Development Engineer, Inference Team - AWS Neuron

Amazon; Seattle, Northern (WA, KY); Hybrid; USD 210,000 - 320,000; Full time

Job summary

Amazon in Seattle is seeking a Sr. Software Development Engineer for the AWS Neuron Inference Team.

You will drive core serving technologies for high-performance model inference across Inferentia/Trainium-powered workloads and work on open-source frameworks to enable scalable deployment of large models. The role requires strong software development experience and system-design leadership, collaborating with model development, compiler, runtime, and performance teams to ship production-ready

Qualifications

• 5+ years of non-internship professional software development experience.
• 5+ years of programming with at least one software programming language experience.
• 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience.
• 5+ years of full software development life cycle, including coding standards.

Responsibilities

• Lead the customization and optimization of open-source inference frameworks (e.g., vLLM, SGLang) on AWS Neuron, including core framework logic such as scheduling and model execution.
• Evaluate emerging inference research and translate into production-ready features, guiding technical roadmap and mentorship.
• Collaborate across model development, compiler, runtime, and performance engineering teams to ensure end-to-end model performance.

Skills

Senior software development

Programming experience

Design/architecture leadership

SDLC

Job description

Sr. Software Development Engineer, Inference Team - AWS Neuron

AWS Neuron is the complete software stack for AWS Inferentia and Trainium, AWS purpose-built accelerators for cloud-scale machine learning. This senior software engineering role is part of the Machine Learning Inference Applications team and focuses on delivering high-performance model inference solutions for customer workloads running on Inferentia- and Trainium-powered instances. The engineer will lead the development of core serving technologies within open-source frameworks (such as vLLM and SGLang) to enable customers to serve large-scale inference efficiently on AWS Neuron. The engineer will also drive excellence in inference framework optimization and engineering best practices. The engineer will work closely with model development engineers, performance engineers, compiler engineers, and runtime engineers to ensure end-to-end model performance and deliver production-ready accuracy, scalability, and efficiency across a broad range of models and customer use cases.

Key job responsibilities

The engineer will lead the customization and optimization of open-source inference frameworks (vLLM, SGLang) on AWS Neuron—including core framework logic such as scheduling and model execution—applying state-of-the-art techniques in kernel development, parallel computation, distributed KV cache, speculative decoding, and systems engineering to deliver best-in-class LLM serving performance. The engineer will also influence the team's technical roadmap by evaluating emerging inference research and translating it into production-ready features, and raise the bar through design leadership and mentorship. This role requires close collaboration across model development, compiler, runtime, and performance engineering teams.

About the team

Our team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge-sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects that help our team members develop your engineering expertise so you feel empowered to take on more complex tasks in the future.

Basic Qualifications

• - 5+ years of non-internship professional software development experience
• - 5+ years of programming with at least one software programming language experience
• - 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience
• - 5+ years of full software development life cycle, including coding standards