Lead Scientist Real-Time Multimodal Conversational AI

Amazon; Boston (MA); On-site; USD 167,000 - 226,000; Full time

Job summary

Amazon is seeking a Senior Applied Scientist to drive research and development of real-time multimodal conversational AI. You will own a significant research area across pre-training, architecture design, and post-training alignment, delivering production-ready models.

The role emphasizes real-time speech/audio generation and interaction, with collaboration with inference engineers to ensure low-latency deployment.

Qualifications

• 5+ years of building machine learning models for business applications.
• PhD or Master’s with 6+ years of applied research experience.
• Experience programming in Java, C++, Python or related language.
• Hands-on experience training large-scale foundation models.
• Experience with multimodal model architectures across speech, text, and audio.

Responsibilities

• Build and train large-scale multimodal foundation models for real-time speech and audio generation.
• Advance scaling and efficiency of conversational models for real-time performance.
• Design architectures informed by hardware constraints and production requirements.
• Develop training methodologies for multimodal models processing speech, language, and audio in real time.
• Design reward models and RL methods to shape conversational behavior.
• Contribute to post-training pipeline from SFT through RL alignment for real-time multimodal outputs.
• Develop evaluation frameworks for latency, audio quality, and natural interaction.

Skills

ML model development

Research leadership

Multimodal models

Transformer architectures

Publication track record

Education

PhD

MS + 6+ years applied research

Tools

Java

C++

Python

TensorFlow/PyTorch

Job description

Amazon is seeking a Senior Applied Scientist to drive research and development of real-time multimodal conversational AI. You will own a significant research area across pre-training, architecture design, and post-training alignment, delivering production-ready models.

The role emphasizes real-time speech/audio generation and interaction, with collaboration with inference engineers to ensure low-latency deployment.