Senior SDE - AI Inference Infra for Global Mantle Platform

Amazon; New York (NY); On-site; USD 168,000 - 227,000; Full time

Benefits offered by this job

Health insurance

401(k) matching

Paid time off

Parental leave

Job summary

Amazon is seeking a Senior Software Development Engineer for the AWS Mantle team in Seattle. You will shape the technical vision for a high-performance distributed inference engine powering Bedrock, targeting millisecond latency, zero-trust security, and scalable global availability.

You will collaborate with senior leadership to align investments, own end-to-end design decisions, and mentor engineers as Mantle scales to onboard new foundation models.

Qualifications

• 5+ years of professional software development experience.
• 5+ years of programming experience in at least one modern language (Java, C++, Python, Go, or Rust).
• Experience leading design or architecture of systems (patterns, reliability, scaling).
• Experience mentoring, tech lead, or engineering leadership.
• Experience delivering cross-organizational strategy in ambiguous environments.

Responsibilities

• Define and execute on the technical vision for Mantle's distributed inference engine.
• Architect scalable, secure, low-latency solutions for global customers.
• Drive cross-team initiatives to deliver OpenAI-compatible API experiences with strong security guarantees.
• Lead strategy for scaling inference to support onboarding of foundation models.
• Mentor engineers and raise the engineering bar across the team.

Skills

Software development

Java

C++

Python

Go

Rust

Distributed systems

Mentorship

Security

Cross-team collaboration

Education

Bachelor's degree in Computer Science/Engineering or related field

Master's degree or PhD in CS/ML/Engineering

Tools

vLLM

TensorRT

Triton Inference Server

Job description

Amazon is seeking a Senior Software Development Engineer for the AWS Mantle team in Seattle. You will shape the technical vision for a high-performance distributed inference engine powering Bedrock, targeting millisecond latency, zero-trust security, and scalable global availability.

You will collaborate with senior leadership to align investments, own end-to-end design decisions, and mentor engineers as Mantle scales to onboard new foundation models.