Senior ML Inference Engineer – High-Performance Serving

Amazon Inc.; Seattle (WA); On-site; USD 168,000 - 227,000; Full time

Job summary

Amazon Inc. in Seattle, WA is seeking a Sr. Software Development Engineer for the Inference Team on AWS Neuron to deliver high-performance model inference for customer workloads on Inferentia- and Trainium-powered instances.

You will lead customization of open-source inference frameworks (vLLM, SGLang) and drive kernel-level and framework optimizations, collaborating across model development, compiler, and runtime teams to ensure scalable, production-ready performance.

Qualifications

• 5+ years of non-internship professional software development experience.
• 5+ years of programming with at least one software programming language experience.
• 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience.
• 5+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience.
• Experience as a mentor, tech lead or leading an engineering team
• Fundamentals of Machine learning models, their architecture, training and inference lifecycles along with work experience on some optimizations for improving the model performance.

Responsibilities

• Lead customization and optimization of open-source inference frameworks (vLLM, SGLang) on AWS Neuron—including core framework logic such as scheduling and model execution—applying state-of-the-art techniques in kernel development, parallel computation, distributed KV cache, speculative decoding, and systems engineering to deliver best-in-class LLM serving performance.
• Influence the team's technical roadmap by evaluating emerging inference research and translating it into production-ready features; provide design leadership and mentorship.
• Collaborate with model development, compiler, runtime, and performance engineering teams to ensure end-to-end model performance and scalable deployments.

Skills

Software development

Programming experience

Design/architecture leadership

SDLC experience

Mentor / tech lead

ML inference basics

Education

Bachelor's degree in CS or equivalent

Tools

PyTorch/JAX

vLLM/SGLang

CUDA/Triton

Job description

Amazon Inc. in Seattle, WA is seeking a Sr. Software Development Engineer for the Inference Team on AWS Neuron to deliver high-performance model inference for customer workloads on Inferentia- and Trainium-powered instances.

You will lead customization of open-source inference frameworks (vLLM, SGLang) and drive kernel-level and framework optimizations, collaborating across model development, compiler, and runtime teams to ensure scalable, production-ready performance.