SDE II: LLM Inference & Kernel Optimization

Amazon Web Services (AWS); Bellevue (WA); On-site; USD 144,000 - 194,000; Full time

Job summary

Amazon Development Center U.S., Inc. in Bellevue, WA is seeking a Software Development Engineer focused on inference performance optimization for LLM workloads.

You will design, implement, and measure kernel-level improvements to reduce latency and boost throughput across production inference systems. This hands-on role requires profiling bottlenecks, writing high-performance code, code reviews, and collaboration with ML engineers and infrastructure teams to deliver production-ready solutions.

Qualifications

• 3+ years of non-internship professional software development experience.
• 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
• 1+ years of software development engineer or related occupational experience
• 1+ years of designing and developing large-scale, multi-tiered, multi-threaded, embedded or distributed software applications, tools, systems, and services using: C#, C++, Java, or Perl experience
• 1+ years of Object Oriented Design experience
• Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field
• Experience programming with at least one software programming language

Responsibilities

• Design and implement kernel-level optimizations and inference pipeline improvements that reduce latency and increase throughput for LLM serving workloads.
• Profile, benchmark, and tune model inference performance across hardware and software stacks, identifying bottlenecks and delivering measurable gains.
• Write well-organized, maintainable code and participate actively in code reviews, providing constructive feedback to peers and mentoring newer engineers on the team.
• Collaborate with applied scientists, ML engineers, and infrastructure teams to translate model requirements into production-ready inference solutions.
• Evaluate emerging tools, frameworks, and generative AI techniques for inference optimization, recommending adoption where they improve efficiency or quality.

Skills

C++

Java

C#

Object Oriented Design

Education

Bachelor's degree in Computer Science, Engineering, Mathematics, or related field

Tools

CUDA

Job description

Amazon Development Center U.S., Inc. in Bellevue, WA is seeking a Software Development Engineer focused on inference performance optimization for LLM workloads.

You will design, implement, and measure kernel-level improvements to reduce latency and boost throughput across production inference systems. This hands-on role requires profiling bottlenecks, writing high-performance code, code reviews, and collaboration with ML engineers and infrastructure teams to deliver production-ready solutions.