Senior SDE — Global ML Inference Engine (Mantle)

Amazon Web Services (AWS); Seattle (WA); On-site; USD 168,000 - 227,000; Full time

Benefits offered by this job

Health insurance

401(k) matching

Paid time off

Parental leave

RSUs

Job summary

Amazon.com Services LLC seeks a Sr. Software Development Engineer to shape Mantle, AWS’s distributed inference engine powering Bedrock. You will influence system design for millisecond latency, high scalability, and robust security across global regions.

You will mentor teams, define long-term architecture, and partner with senior leadership to align investments with business goals while delivering reliable AI infrastructure at scale.

Qualifications

• 5+ years of non-internship professional software development experience.
• 5+ years of leading design or architecture of new and existing systems.
• Experience mentoring, tech lead or leading an engineering team.
• Bachelor’s degree in CS, Engineering or related field, or equivalent experience.
• 5+ years of programming with Java, C++, Python, Go, or Rust.
• Experience driving cross-organizational technical strategy in complex, ambiguous environments.

Responsibilities

• As a Sr. SDE on Mantle, act as technical conscience and strategic leader for a critical AI infrastructure platform.
• Architect Mantle's distributed inference engine, including capacity management and QoS controls.
• Drive cross-organizational initiatives to deliver OpenAI-compatible API experiences with ZOA security.
• Lead strategy to scale inference for rapid onboarding of foundation models with global availability.

Skills

Distributed systems

ML infrastructure

Java

C++

Python

Go

Education

Bachelor's degree in Computer Science or related field

Master's degree preferred

Tools

vLLM

TensorRT

Triton Inference Server

Job description

Amazon.com Services LLC seeks a Sr. Software Development Engineer to shape Mantle, AWS’s distributed inference engine powering Bedrock. You will influence system design for millisecond latency, high scalability, and robust security across global regions.

You will mentor teams, define long-term architecture, and partner with senior leadership to align investments with business goals while delivering reliable AI infrastructure at scale.