Real-Time Multimodal Inference Engineer

Amazon; Sunnyvale (CA); On-site; USD 165,000 - 224,000; Full time

Benefits offered by this job

Health insurance

401(k) matching

Paid time off

Parental leave

Job summary

Amazon is seeking an Inference Engineer in Sunnyvale to own components of the inference stack and drive their execution with guidance from scientists and senior engineers. You will shape model architectures for servability, build real-time runtimes, and develop offline systems for training and reinforcement learning."

You will collaborate across teams to ensure neural models run within hard latency budgets on production hardware, balancing throughput and cost at scale.

Qualifications

• Master's degree or higher in CS or related field.
• 3+ years of professional software development experience.
• 2+ years in design/architecture of new/existing systems.
• 1+ years optimizing inference for neural models.
• Deep learning architectures (transformers) and multimodal models.

Responsibilities

• Help make model architectures servable with latency, memory, and cost focus.
• Implement and optimize inference paths for large-scale multimodal models.
• Develop high-performance kernels for critical operations in production serving.
• Profile end-to-end performance and identify bottlenecks in large-scale inference workloads.
• Support real-time streaming serving with sub-second latency under concurrent load.

Skills

Latency optimization

GPU performance

Distributed systems

Real-time inference

Python

Education

Master's degree

Bachelor's degree in CS

Tools

vLLM

TensorRT-LLM

CUDA

Triton

Job description

Amazon is seeking an Inference Engineer in Sunnyvale to own components of the inference stack and drive their execution with guidance from scientists and senior engineers. You will shape model architectures for servability, build real-time runtimes, and develop offline systems for training and reinforcement learning."

You will collaborate across teams to ensure neural models run within hard latency budgets on production hardware, balancing throughput and cost at scale.