Lead Real-Time Multimodal AI Scientist, Conversational/AGI

Amazon; Bellevue (WA); On-site; USD 180,000 - 260,000; Full time

Job summary

Amazon is seeking a Senior Applied Scientist to drive the research and development of real-time multimodal conversational AI, focusing on foundation models for speech and audio and post-training systems shaping human-like conversational behavior.

You will own a major research area across pre-training, architecture design through post-training alignment and real-time deployment, collaborating with inference engineers to ensure real-time production readiness.

Qualifications

• 5+ years of building machine learning models for business applications.
• PhD or MS with 6+ years of applied research experience.
• Experience programming in Java, C++, Python or related language.
• Hands-on experience training large-scale foundation models.
• Experience with multimodal model architectures across speech, text, and audio.
• Strong understanding of transformer architectures in speech/audio.
• Top-tier publication record (NeurIPS, ICML, ICLR, Interspeech, ICASSP, ACL).
• Demonstrated ownership of a research area and collaboration across scientists and engineers.

Responsibilities

• Build and train large-scale multimodal foundation models for real-time speech and audio generation.
• Advance scaling and efficiency of conversational models relating data, size, and real-time performance.
• Design architectures informed by hardware constraints and inference requirements.
• Develop training methodologies for multimodal models processing speech, language, and audio in real-time.
• Contribute to state-of-the-art efficient architectures and training methods for conversational AI at scale.
• Design and implement reward models and reinforcement learning methods to shape conversational behavior.
• Build parts of the post-training pipeline from SFT through RL alignment for real-time multimodal outputs.
• Design evaluation frameworks capturing latency, audio quality, prosody, and interaction naturalness in real-time conversations.
• Advance real-time perception: process incoming audio while generating responses.
• Develop techniques for natural interactive systems with concurrent input/output and human-like timing.

Skills

Java

C++

Python

Multimodal models

Transformer architectures

Publication record

Ownership of research area

Mentoring

Education

PhD or MS + 6+ years applied research

Job description

Amazon is seeking a Senior Applied Scientist to drive the research and development of real-time multimodal conversational AI, focusing on foundation models for speech and audio and post-training systems shaping human-like conversational behavior.

You will own a major research area across pre-training, architecture design through post-training alignment and real-time deployment, collaborating with inference engineers to ensure real-time production readiness.