Open-Source ML Inference Engineer for Neuron

Amazon Inc.; Seattle (WA); On-site; USD 144,000 - 194,000; Full time

Benefits offered by this job

RSUs

Health insurance

401(k) matching

Paid time off

Parental leave

Job summary

Amazon AWS Neuron team is seeking a Software Engineer to advance AI/ML inference on Neuron chips. You will contribute to serving tech for large-scale model inference, collaborating on frameworks like vLLM and SGLang to accelerate model deployment and production readiness.

Work with cross-functional teams to optimize performance, reliability, and scalability, delivering end-to-end model support across diverse workloads and ensuring high-quality engineering practices.

Qualifications

• 3+ years of non-internship professional software development experience.

Responsibilities

• Contribute core serving features to vLLM and SGLang, enabling continuous batching, quantization, and distributed inference on Neuron.

Skills

C++

C#

Java

Perl

Transformer architecture

LLM fundamentals

Debugging

Education

Bachelor's degree in CS/Engineering/Math

Tools

vLLM

SGLang

TensorRT

CUDA

Triton

Job description

Amazon AWS Neuron team is seeking a Software Engineer to advance AI/ML inference on Neuron chips. You will contribute to serving tech for large-scale model inference, collaborating on frameworks like vLLM and SGLang to accelerate model deployment and production readiness.

Work with cross-functional teams to optimize performance, reliability, and scalability, delivering end-to-end model support across diverse workloads and ensuring high-quality engineering practices.