Senior ML Inference Engineer — Open-Source LLM Serving

Amazon

Seattle, Northern (WA, KY)

Hybrid

USD 210,000 - 320,000

Full time

Just now

Be an early applicant

Application generator

Stand out for this role — generate a tailored resume and cover letter in about a minute.

Get past ATS filters

Job summary

Amazon in Seattle is seeking a Sr. Software Development Engineer for the AWS Neuron Inference Team.

You will drive core serving technologies for high-performance model inference across Inferentia/Trainium-powered workloads and work on open-source frameworks to enable scalable deployment of large models. The role requires strong software development experience and system-design leadership, collaborating with model development, compiler, runtime, and performance teams to ship production-ready

Qualifications

5+ years of non-internship professional software development experience.
5+ years of programming with at least one software programming language experience.
5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience.
5+ years of full software development life cycle, including coding standards.

Responsibilities

Lead the customization and optimization of open-source inference frameworks (e.g., vLLM, SGLang) on AWS Neuron, including core framework logic such as scheduling and model execution.
Evaluate emerging inference research and translate into production-ready features, guiding technical roadmap and mentorship.
Collaborate across model development, compiler, runtime, and performance engineering teams to ensure end-to-end model performance.