ML Inference Engineer — High-Performance LLM Serving

Amazon Inc.; Bellevue (WA); On-site; USD 144,000 - 194,000; Full time

Benefits offered by this job

Health insurance

401(k) matching

Paid time off

Parental leave

Job summary

Amazon in Bellevue, WA is seeking a Software Development Engineer to optimize inference performance and serve efficiency for large language models. You’ll design kernel optimizations, profile across hardware/software stacks, and ship production‑readiness inference solutions.

You will mentor peers and collaborate with scientists and infrastructure teams daily. The role emphasizes hands‑on performance work, profiling bottlenecks, and shipping effective optimizations that reduce latency and boost

Qualifications

• 3+ years of non‑internship software development experience.
• 2+ years of design or architecture (design patterns, reliability and scaling) of new and existing systems.
• 1+ years of software development engineer or related occupational experience.
• 1+ years of designing and developing large‑scale, multi‑tiered, multi‑threaded, embedded or distributed software applications, tools, systems, and services using C#, C++, Java, or Perl.
• 1+ years of Object Oriented Design experience.
• Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field.

Responsibilities

• Design and implement kernel‑level optimizations and inference pipeline improvements that reduce latency and increase throughput for LLM serving workloads.
• Profile, benchmark, and tune model inference performance across hardware and software stacks, identifying bottlenecks and delivering measurable gains.
• Write well‑organized, maintainable code and participate actively in code reviews, providing constructive feedback to peers and mentoring newer engineers on the team.
• Collaborate with applied scientists, ML engineers, and infrastructure teams to translate model requirements into production‑ready inference solutions.
• Evaluate emerging tools, frameworks, and generative AI techniques for inference optimization, recommending adoption where they improve efficiency or quality.

Skills

C++

Java

C#

Perl

Multithreading

Object Oriented Design

Education

Bachelor's degree in CS/Engineering/Math

Tools

CUDA kernels

Job description

Amazon in Bellevue, WA is seeking a Software Development Engineer to optimize inference performance and serve efficiency for large language models. You’ll design kernel optimizations, profile across hardware/software stacks, and ship production‑readiness inference solutions.

You will mentor peers and collaborate with scientists and infrastructure teams daily. The role emphasizes hands‑on performance work, profiling bottlenecks, and shipping effective optimizations that reduce latency and boost