Real-Time Multimodal Inference Engineer

Amazon; Seattle (WA); On-site; USD 144,000 - 194,000; Full time

Benefits offered by this job

Health insurance

401(k) matching

Paid time off

Parental leave

Job summary

Amazon in Seattle seeks an Inference Engineer to own components of real-time multimodal inference, from research to production, ensuring sub-second latency within cost constraints.

You will collaborate with scientists and senior engineers to optimize the inference stack, develop high-performance kernels, and shape model architectures to be servable. This role blends research and production in a fast-paced environment.

Qualifications

• Master's degree or equivalent.
• 3+ years non-internship software development experience.
• 3+ years programming in at least one language.
• 2+ years design/architecture experience.
• 1+ years optimizing neural inference.
• Solid understanding of transformers and speech/audio applications.
• Experience with latency-constrained real-time inference.
• Experience with GPU performance optimization.

Responsibilities

• Partner with researchers to help make model architectures servable and surface latency and cost implications.
• Implement and optimize parts of the inference path for large-scale multimodal models and KV-cache.
• Apply efficiency techniques across the stack and measure latency trade-offs.
• Develop high-performance kernels and integrate into production serving.
• Profile end-to-end performance using Nsight Compute/Systems and roofline analysis.
• Contribute to real-time streaming serving paths and latency budgets.
• Build offline inference systems for RL/RLHF/RLAIF and train/serve parity.

Skills

Software dev

Programming languages

Systems design

GPU optimization

Real-time inference

Ownership

Education

Master's degree or equivalent

Bachelor's degree in CS or equivalent

Tools

vLLM

TensorRT-LLM

CUTLASS

Triton

FlashAttention

KV-cache

Job description

Amazon in Seattle seeks an Inference Engineer to own components of real-time multimodal inference, from research to production, ensuring sub-second latency within cost constraints.

You will collaborate with scientists and senior engineers to optimize the inference stack, develop high-performance kernels, and shape model architectures to be servable. This role blends research and production in a fast-paced environment.