Senior Real-Time Multimodal Inference Engineer

Amazon; Seattle (WA); On-site; USD 167,000 - 226,000; Full time

Benefits offered by this job

Health insurance

401(k) matching

Paid time off

Parental leave

RSUs

Job summary

Amazon in Seattle is seeking a Senior Inference Engineer to own end-to-end real-time inference for multimodal models, spanning research to production. You will shape architectures for servability, build streaming runtimes, and develop offline systems for RL and post-training rollout.

You will collaborate with scientists and hardware partners to achieve sub-second latency on real-time workloads while controlling cost and ensuring scalability across distributed systems and multiple GPUs.

Qualifications

• 3+ years building ML models for business use cases.
• PhD or MS with 6+ years of applied research.
• Experience programming in Java, C++, Python or related language.
• Hands-on experience optimizing inference for neural models and profiling.

Responsibilities

• Partner with researchers to surface latency, memory, and cost implications early.
• Implement and optimize inference paths for large multimodal models with low latency.
• Apply efficiency techniques like quantization and decoding trade-offs.
• Develop high-performance kernels and integrate into production serving.
• Profile end-to-end performance with Nsight Compute/Systems and roofline analysis.
• Own real-time streaming serving path and meet sub-second latency under load.
• Brandish continuous batching, scheduling, and preemption for throughput vs latency.
• Customize production serving frameworks for real-time streaming generative models.
• Implement multi-GPU inference and drive cost towards production baselines.
• Publish latency, throughput, and cost benchmarks to gate deployment.

Skills

ML model building

Java/C++/Python

Inference optimization

GPU performance

Production scale delivery

Education

PhD or MS + 6y

Tools

vLLM

PyTorch

TensorRT-LLM

NCCL

Job description

Amazon in Seattle is seeking a Senior Inference Engineer to own end-to-end real-time inference for multimodal models, spanning research to production. You will shape architectures for servability, build streaming runtimes, and develop offline systems for RL and post-training rollout.

You will collaborate with scientists and hardware partners to achieve sub-second latency on real-time workloads while controlling cost and ensuring scalability across distributed systems and multiple GPUs.