Research Scientist Intern: Audio-Visual Conversation Gen

Redmond

On-site

AUD 72,000 - 116,000

Full time

12 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Meta is seeking a Reality Labs Research Audio intern to help generate realistic video with audio of multi-person conversations. The role involves controlling 3D scene positions, gaze and turn-taking within egocentric camera setups to create accurately labelled data for audio-visual models.

The candidate should be pursuing a PhD in CS/EE/ML or related fields and have strong research experience in generative video/image modeling, with publications at major venues and hands-on diffusion or

Qualifications

Currently has or is pursuing a PhD in Computer Science, Electrical Engineering, Machine Learning or a related field.
Research experience in generative modeling for video or images with first-author publications at CVPR, ICCV, ECCV, NeurIPS, ICLR or SIGGRAPH.
Hands-on experience training or fine-tuning modern video diffusion or flow-matching models (e.g. DiT-based architectures).
Experience with controllable generation from geometric signals: camera pose, 3D layout, depth or human pose.
Solid grounding in 3D vision: camera models, multi-view geometry and coordinate transforms.
3+ years experience with Python, Matlab, PyTorch, large-scale distributed GPU training or similar.
Experience with joint audio-video generation, speech-driven talking heads or gestures, multi-speaker TTS, or lip sync.
Background in spatial audio: binaural rendering, HRTFs, or room acoustics simulation.
Research on modeling social interaction: conversational dynamics, gaze, turn-taking or group formations.
Familiarity with egocentric perception and datasets (Ego4D, Ego-Exo4D, Aria) and related tasks.
Experience with 3D or 4D human and scene representations (SMPL-X, Gaussian splatting, NeRF) or multi-view-consistent video generation.
Experience with synthetic data generation, game engines, or improving model performance on real data.
Experience using LLMs or VLMs to plan scenes, layouts or dialogue.
Track record of open-source contributions or a previous industry research internship.
Proven track record of results shown in accepted papers at top conferences like CVPR, ECCV, NeurIPS, ICASSP, InterSpeech.
Experience working and communicating cross-functionally in a team.
Intent to return to a degree-program after completion of internship/co-op.

Responsibilities

Generate realistic video with audio of multi-person conversations with controllable 3D positioning.
Develop scenes with natural social interactions: gaze, turn-taking and group formations.
Prepare labelled training data for audio-visual models using egocentric camera setups.
Collaborate with researchers, engineers and product teams on project goals.