Senior ML Engineer - Generative AI Inference (SageMaker)

Amazon Inc.; Bellevue (WA); On-site; USD 168,000 - 227,000; Full time

Job summary

Amazon.com Services LLC in Bellevue, WA is seeking a Senior Machine Learning Engineer for SageMaker Inference. You will design and optimize kernel-level inference, profile across hardware, and collaborate with scientists and infra teams to deliver scalable ML serving solutions.

The role emphasizes low latency, high throughput, and production-grade engineering practices in a fast-growing AWS service. You will mentor engineers, participate in code reviews, and shape platform evolution to support

Qualifications

• 5+ years of professional software development experience.
• Experience leading design or architecture of large systems.
• Experience mentoring or leading an engineering team.
• Experience with ML inference and optimizations is preferred.

Responsibilities

• Design and implement kernel-level optimizations and inference pipeline improvements to reduce latency and increase throughput for LLM serving workloads.
• Profile, benchmark, and tune model inference performance across hardware and software stacks.
• Write well-organized, maintainable code and participate in code reviews, mentoring newer engineers.
• Collaborate with applied scientists, ML engineers, and infrastructure teams to translate model requirements into production-ready inference solutions.
• Evaluate emerging tools and techniques for inference optimization and adoption.

Skills

Software development

Leadership

System design

CUDA kernels

Tools

CUDA

LLM inference

Profiling tools

Job description

Amazon.com Services LLC in Bellevue, WA is seeking a Senior Machine Learning Engineer for SageMaker Inference. You will design and optimize kernel-level inference, profile across hardware, and collaborate with scientists and infra teams to deliver scalable ML serving solutions.

The role emphasizes low latency, high throughput, and production-grade engineering practices in a fast-growing AWS service. You will mentor engineers, participate in code reviews, and shape platform evolution to support