Lead Real-Time Multimodal AI Scientist

Amazon; Sunnyvale (CA); On-site; USD 229,000 - 309,000; Full time

Job summary

Amazon is seeking a Principal Applied Scientist to drive research and development of real-time multimodal conversational AI in Sunnyvale, CA. You will advance foundation models for speech and audio and shape post-training systems (reward modeling and reinforcement learning) for natural, human-like interactions.

You will set technical direction across the full model lifecycle—from pre-training and architecture design to RL alignment and real-time deployment—collaborating with inference engineers

Qualifications

• PhD in Electrical Engineering, Computer Science, Mathematics or related field (PhD required).
• 10+ years building speech recognition and NLP systems or related domains.
• Hands-on experience scaling large-scale foundation models.
• Experience with multimodal model architectures (speech/text/audio).
• Strong understanding of transformer architectures in speech/audio domains.
• Top-tier publication record (NeurIPS/ICML/ICLR/Interspeech/ICASSP/ACL).
• Proven Principal/Staff+ scientific leadership and mentoring.

Responsibilities

• Build and train large multimodal foundation models for real-time speech and audio generation from architecture design to production-scale training.
• Advance scaling and efficiency of conversational models balancing data, size, and real-time performance.
• Design architectures aligned with hardware constraints for real-time inference.
• Develop training methods for multimodal models that process speech, language, and audio in streaming contexts.
• Design and build reward models and RL methods to shape natural and responsive conversation.
• Create post-training pipelines from SFT through RL alignment optimized for real-time multimodal outputs.
• Develop evaluation frameworks capturing real-time conversational quality and latency.

Skills

Multimodal models

Reinforcement learning

RLHF/RLAIF

Real-time systems

Speech & audio

Transformer architectures

Research leadership

Education

PhD in Electrical Engineering, Computer Science, Mathematics or related field

Tools

Open-source ML systems

Job description

Amazon is seeking a Principal Applied Scientist to drive research and development of real-time multimodal conversational AI in Sunnyvale, CA. You will advance foundation models for speech and audio and shape post-training systems (reward modeling and reinforcement learning) for natural, human-like interactions.

You will set technical direction across the full model lifecycle—from pre-training and architecture design to RL alignment and real-time deployment—collaborating with inference engineers