Software Development Engineer II, SageMaker AI Inference

Jobverse.io; Bellevue (NE); On-site; USD 140,000 - 200,000; Full time

Job summary

Amazon is hiring a Software Development Engineer II in Bellevue, NE to advance SageMaker AI inference performance. The role centers on reducing latency and increasing throughput for LLM serving through kernel tuning, profiling, and building production-ready inference systems.

You will design and implement kernel-level optimizations, profile across hardware/software stacks, and mentor engineers while collaborating with scientists and infrastructure teams to translate requirements into scalable

Responsibilities

• Design and implement kernel-level optimizations to reduce latency and increase throughput for LLM serving workloads.
• Profile, benchmark, and tune model inference performance across hardware and software stacks, identifying bottlenecks and delivering measurable gains.
• Write well-organized, maintainable code and participate in code reviews, mentoring newer engineers on the team.
• Collaborate with applied scientists, ML engineers, and infra teams to translate model requirements into production-ready inference solutions.
• Evaluate emerging tools and generative AI techniques for inference optimization, recommending adoption when beneficial.

Job description

Amazon is hiring a Software Development Engineer II for SageMaker AI inference performance optimization. The role focuses on improving LLM inference latency and throughput through kernel tuning, performance profiling, and production-ready inference systems.

Responsibilities

• Design and implement kernel-level optimizations and inference pipeline improvements that reduce latency and increase throughput for LLM serving workloads
• Profile, benchmark, and tune model inference performance across hardware and software stacks, identifying bottlenecks and delivering measurable gains
• Write well-organized, maintainable code and participate actively in code reviews, providing constructive feedback to peers and mentoring newer engineers on the team
• Collaborate with applied scientists, ML engineers, and infrastructure teams to translate model requirements into production-ready inference solutions
• Evaluate emerging tools, frameworks, and generative AI techniques for inference optimization, recommending adoption where they improve efficiency or quality