AI Inference Engineer - Neuron & Open-Source Serving

Amazon Web Services (AWS); Seattle (WA); On-site; USD 144,000 - 194,000; Full time

Job summary

Annapurna Labs (U.S.) Inc. in Seattle seeks an experienced software engineer to join the AWS Neuron serving team. You will contribute core features for open-source inference frameworks and optimize model inference on Neuron chips.

You will collaborate with ML, compiler and runtime engineers to deliver scalable, production-ready performance across models and workloads, while adhering to engineering best practices and delivering reliable code.

Qualifications

• 3+ years of professional software development experience.
• 3+ years of design/architecture experience on large systems.
• 2+ years designing multi-tiered, distributed software using C#/C++/Java/Perl.

Responsibilities

• Contribute core serving features to open-source inference frameworks (vLLM, SGLang) and upstream support for continuous batching, quantization, and distributed inference on Neuron.
• Broaden the models supported and build tooling to shorten path from model to production deployment.
• Improve development and shipping velocity by strengthening tests, benchmarks, and release workflows.
• Collaborate with model development, performance, compiler, and runtime engineers to deliver end-to-end model performance across workloads.
• Apply engineering best practices to ship reliable, high-performance inference for customers.

Skills

Software development

System design

Large-scale software

ML/LLM basics

Debugging & profiling

LLM deployment

vLLM/SGLang/TensorRT

CUDA/Triton kernel

Education

Bachelor's degree in CS/Engineering/Math

Job description

Annapurna Labs (U.S.) Inc. in Seattle seeks an experienced software engineer to join the AWS Neuron serving team. You will contribute core features for open-source inference frameworks and optimize model inference on Neuron chips.

You will collaborate with ML, compiler and runtime engineers to deliver scalable, production-ready performance across models and workloads, while adhering to engineering best practices and delivering reliable code.