Senior Real-Time Multimodal Inference Engineer

Amazon; Sunnyvale (CA); On-site; USD 193,000 - 262,000; Full time

Benefits offered by this job

Health insurance

401(k) matching

Paid time off

Parental leave

Job summary

Amazon is seeking a Senior Inference Engineer to own the end-to-end real-time multimodal inference stack. You will shape model architectures for servability, build real-time runtimes, and scale offline systems for RL and evaluation.

The role spans research-to-production, requiring deep knowledge of transformers, KV-cache, and GPU optimization. You will collaborate with scientists and hardware partners to deliver sub-second latency under concurrent load, while balancing performance and cost.

Qualifications

• 5+ years of non-internship professional software development experience.
• 5+ years of programming with at least one software programming language experience.
• 4+ years of leading design or architecture of new and existing systems.
• Bachelor's degree in computer science or equivalent.
• 2+ years of hands-on experience optimizing inference for neural models.
• Strong understanding of deep learning architectures and their application to speech/audio or other multimodal domains.
• Production track record delivering latency-constrained, real-time inference systems under concurrent load.
• Experience with GPU performance optimization and KV-cache management.

Responsibilities

• Own inference stack end to end for real-time multimodal models.
• Design and optimize the real-time streaming serving path with sub-second latency.
• Develop offline RL/evaluation pipelines for model improvements.
• Collaborate with scientists and hardware partners to ensure scalable, cost-effective serving.

Skills

Deep learning architectures

Transformer models

Real-time inference

Latency optimization

Education

Bachelor's degree in computer science or equivalent

Tools

CUTLASS

Triton

CUDA/PTX

vLLM

TensorRT-LLM

Job description

Amazon is seeking a Senior Inference Engineer to own the end-to-end real-time multimodal inference stack. You will shape model architectures for servability, build real-time runtimes, and scale offline systems for RL and evaluation.

The role spans research-to-production, requiring deep knowledge of transformers, KV-cache, and GPU optimization. You will collaborate with scientists and hardware partners to deliver sub-second latency under concurrent load, while balancing performance and cost.