Senior Real-Time Multimodal Inference Architect

Amazon; Sunnyvale (CA); On-site; USD 192,000 - 260,000; Full time

Benefits offered by this job

Health insurance

401(k) matching

RSU/Stock options

Job summary

Amazon is seeking a Senior Inference Engineer in Sunnyvale, CA to own the end-to-end inference stack for real-time multimodal conversational AI. You will shape model architectures, build low-latency streaming serving paths, and develop offline training and RL/evaluation infrastructure.

Expect cross-team collaboration with scientists and hardware partners to ensure scalable, cost-effective performance. You will drive architecture co-design, optimize large-scale inference, and implement

Qualifications

• 3+ years building ML models for business applications
• PhD or MS + 6+ years research experience
• Experience programming in Java, C++, Python or related language
• 2+ years optimizing inference for neural models
• Strong understanding of transformers and autoregressive decoding
• Production latency-constrained inference experience

Responsibilities

• Own end-to-end inference stack from research to production
• Optimize real-time streaming inference for multimodal models
• Implement low-latency serving paths and multi-GPU inference
• Develop offline training and RL evaluation infrastructure

Skills

Machine learning inference

Java/C++/Python

Latency optimization

Profiling

Multimodal models

Real-time systems

Ocean of compute

Education

PhD or MS + 6+ yrs research

Tools

Nsight Compute

PyTorch

TensorRT

Job description

Amazon is seeking a Senior Inference Engineer in Sunnyvale, CA to own the end-to-end inference stack for real-time multimodal conversational AI. You will shape model architectures, build low-latency streaming serving paths, and develop offline training and RL/evaluation infrastructure.

Expect cross-team collaboration with scientists and hardware partners to ensure scalable, cost-effective performance. You will drive architecture co-design, optimize large-scale inference, and implement