Senior Real-Time Multimodal Inference Engineer

Amazon; Boston, Northern (MA, KY); Hybrid; USD 167,000 - 226,000; Full time

Job summary

Amazon.com Services LLC is seeking a Senior Inference Engineer to own the inference stack for real-time multimodal conversational AI. You will shape model architectures for servable deployment, build real-time runtimes within tight latency budgets, and develop offline systems for training and RL integration.

You will collaborate with scientists and hardware teams to ensure fast, cost-effective inference at scale, across training, evaluation, and deployment pathways.

Qualifications

• 3+ years of building ML models for business applications.
• PhD, or Master's with 6+ years of applied research experience.
• Experience with Java, C++, Python or related language.
• 2+ years hands-on experience optimizing inference for neural models.
• Strong understanding of transformers, attention, and autoregressive decoding.
• Production track record delivering latency-constrained real-time inference.

Responsibilities

• Own the real-time streaming inference path for multimodal models.
• Develop and optimize the end-to-end inference stack from research to production.
• Profile and optimize latency, memory, and cost across the stack.
• Collaborate with scientists and hardware partners to meet latency budgets.
• Establish performance benchmarks and publish operational metrics.

Skills

Latency optimization

Real-time inference

Multimodal models

System design

Collaboration

Education

PhD or Master's + research

Tools

Java

C++

Python

Nsight Compute

TensorRT-LLM

vLLM

CUDA

Job description

Amazon.com Services LLC is seeking a Senior Inference Engineer to own the inference stack for real-time multimodal conversational AI. You will shape model architectures for servable deployment, build real-time runtimes within tight latency budgets, and develop offline systems for training and RL integration.

You will collaborate with scientists and hardware teams to ensure fast, cost-effective inference at scale, across training, evaluation, and deployment pathways.