Senior Real-Time Multimodal Inference Engineer

Amazon Inc.; Boston (MA); On-site; USD 167,000 - 226,000; Full time

Job summary

Amazon.com Services LLC is hiring a Senior Inference Engineer to own real-time multimodal inference across research to production. You will co-design architectures, optimize the streaming path, and build offline training/evaluation infra for large-scale models.

The role requires 3+ years of ML model experience, strong software skills, and a track record delivering latency-constrained, real-time systems with GPU optimization. Boston base, full-time.

Qualifications

• 3+ years building ML models for business applications.
• PhD or Master's with 6+ years of applied research experience.
• Experience programming in Java, C++, Python or related language.
• 2+ years of hands-on experience optimizing inference for neural models.

Responsibilities

• Model Architecture & Inference Co-Design: surface latency, memory, and cost implications.
• Real-Time & Interactive Runtime: meet sub-second streaming latency under load.
• Offline Systems: training, RL, and evaluation infrastructure behind post-training.
• Own end-to-end inference stack areas and drive technical execution.
• Collaborate with scientists and hardware partners to optimize performance.

Skills

Java

C++

Python

Education

PhD or MSc

Tools

vLLM

PyTorch

TensorRT-LLM

CUTLASS

Job description

Amazon.com Services LLC is hiring a Senior Inference Engineer to own real-time multimodal inference across research to production. You will co-design architectures, optimize the streaming path, and build offline training/evaluation infra for large-scale models.

The role requires 3+ years of ML model experience, strong software skills, and a track record delivering latency-constrained, real-time systems with GPU optimization. Boston base, full-time.