Research Scientist - World Modeling

Mohamed bin Zayed University of Artificial Intelligence; Sunnyvale (CA); Sur place; USD 150 000 - 450 000; Plein temps

Avantages offerts par ce poste

Medical, dental, and vision

Performance bonus

401K Plan

Paid time off

Parental leave

Employee assistance program

Life insurance and disability

Résumé du poste

MBZUAI is seeking a senior researcher to advance the PAN world models within the AllWorld team. You will lead large-scale training on massive clusters, collaborate with engineers and data scientists, and push the frontiers of video generation, multimodal models, and world modeling research.

You will develop scalable training pipelines, design robust evaluation benchmarks, optimize inference for real-time interaction, and contribute to publications and technology transfer in a fast-paced research

Qualifications

• MSc or PhD in machine learning or CS or equivalent industry experience.
• Experience in training large-scale models (LLMs or diffusion) on large clusters.
• Hands-on experience with state-of-the-art video generative models (e.g., Sora, Veo2, MovieGen, CogVideoX).
• Experience building and optimizing large-scale video data pipelines.
• Experience accelerating diffusion model inference for improved efficiency.
• Strong systems and engineering expertise in PyTorch.
• Excellent problem-solving and collaboration skills.
• Research contributions to top-tier conferences or journals preferred.

Responsabilités

• Develop the foundational world model to simulate the physical world.
• Collaborate with engineering and data teams on large-scale training.
• Develop metrics and evaluation benchmarks to assess model performance.
• Design scalable data annotation pipelines for training and evaluation.
• Optimize inference to enable real-time interaction.
• Advance scalable training systems for multimodal LLMs and video diffusion models.
• Build efficient video data pipelines and annotation frameworks.
• Enhance inference efficiency through distillation techniques.

Connaissances

Large-scale training

PyTorch

Systems engineering

Problem-solving

Communication

Formation

MSc or PhD in ML/CS

Outils

Sora

Veo2

MovieGen

CogVideoX

Description du poste

About the Institute of Foundation Models

We are a dedicated research lab for building, understanding, using,and risk-managing foundation models. Our mandate is to advanceresearch, nurture the next generation of AI builders, and drivetransformative contributions to a knowledge-driven economy.

As part of our team, you’ll have the opportunity to work on thecore of cutting-edge foundation model training, alongsideworld-class researchers, data scientists, and engineers, tacklingthe most fundamental and impactful challenges in AIdevelopment. You will participate in the development ofgroundbreaking AI solutions that have the potential to reshapeentire industries. Strategic and innovative problem-solving skillswill be instrumental in establishing MBZUAI as a global hub forhigh-performance computing in deep learning, driving impactfuldiscoveries that inspire the next generation of AI pioneers.

The Role

We are the AllWorld Team under the Institute ofFoundation Model (IFM) at MBZUAI . At AllWorld, we arepioneering the development of the PAN (Physical, Agentic, andNetworked) world models —the next-generation foundation modelsto unlock machine intelligence beyond lingual.

Our mission is to tackle the fundamental challenges of worldmodeling and establish a new paradigm for next-generationmachine reasoning . We are looking for passionate individualswho share our vision and are eager to push the boundaries of AItogether.

Key Responsibilities

• Develop the foundational world model to accurately simulate the physical world.
• Collaborate with engineering and data teams to tackle key challenges in training the world model on large-scale clusters.
• Develop metrics and evaluation benchmarks to better assess model performance.
• Design and implement a scalable and efficient data annotation pipeline to ensure high-quality labeled data for training and evaluation.
• Optimize inference efficiency to enable real-time interaction.

Areas of Focus

• Scalable Training Systems : Develop and optimize infrastructure for training multimodal LLMs and video diffusion models at massive scale.
• Efficient Data Pipelines : Build scalable video data pipelines and annotation frameworks to support high-quality training data.
• Inference Optimization : Enhance inference efficiency through optimization and distillation techniques to enable real-time interaction.
• Visual Tokenization : Develop methods for discretizing visual features into tokens for improved model representation.
• Quantitative Evaluation : Establish rigorous benchmarks to assess physical accuracy, controllability, and intelligence.
• Scaling Laws for Video Pretraining : Investigate scaling law principles to guide efficient video pre-training strategies.

Academic Qualifications

• MSc or PhD in Machine Learning or Computer Science, or equivalent industry experience.

Professional Experience

• Experience in large-scale model training (LLMs or Diffusion Models) on large clusters.
• Hands-on experience with state-of-the-art video generative models (e.g., Sora, Veo2, MovieGen, CogVideoX, etc.).
• Experiences in building and optimizing large-scale video data pipelines.
• Experience in accelerating diffusion model inference for improved efficiency.
• Exceptional problem-solving and troubleshooting skills to tackle complex technical challenges.
• Strong systems and engineering expertise in deep learning frameworks such as PyTorch.
• Strong communication and collaboration skills for effective cross-functional teamwork.
• Ability to navigate ambiguity and drive projects in rapidly evolving research areas.
• Research contributions to top-tier conferences or journals (e.g., ICML, ICLR, NeurIPS, ACL, CVPR, COLM, etc.), with published work in relevant domains.

$150,000 - $450,000 a year

Compensation

$150,000 – $450,000/year + discretionary performance bonus + benefits

The posted salary range represents the company’s good faithestimate of the base compensation for this position upon hire. Theactual base compensation offered may vary within this rangedepending on individual qualifications, including but not limitedto relevant skills, experience, education, certifications, andspecific business needs. In addition to base salary, totalcompensation includes an annual discretionary performance bonus andcomprehensive benefits.

Benefits Include

• Comprehensive medical, dental, and vision benefits
• Bonus
• 401K Plan
• Generous paid time off, sick leave and holidays
• Paid Parental Leave
• Employee Assistance Program
• Life insurance and disability

We may use artificial intelligence (AI) tools to support parts ofthe hiring process, such as reviewing applications, analyzingresumes, or assessing responses and identifying potentialinconsistencies or verification signals in application materialsbased on available information. These tools assist our recruitmentteam but do not replace human judgment. Final hiring decisions areultimately made by humans. If you would like more information abouthow your data is processed, please contact us.

About the Institute of Foundation Models

We are a dedicated research lab for building, understanding, using,and risk-managing foundation models. Our mandate is to advanceresearch, nurture the next generation of AI builders, and drivetransformative contributions to a knowledge-driven economy.

As part of our team, you’ll have the opportunity to work on thecore of cutting-edge foundation model training, alongsideworld-class researchers, data scientists, and engineers, tacklingthe most fundamental and impactful challenges in AIdevelopment. You will participate in the development ofgroundbreaking AI solutions that have the potential to reshapeentire industries. Strategic and innovative problem-solving skillswill be instrumental in establishing MBZUAI as a global hub forhigh-performance computing in deep learning, driving impactfuldiscoveries that inspire the next generation of AI pioneers.

The Role

We are the AllWorld Team under the Institute ofFoundation Model (IFM) at MBZUAI . At AllWorld, we arepioneering the development of the PAN (Physical, Agentic, andNetworked) world models —the next-generation foundation modelsto unlock machine intelligence beyond lingual.

Our mission is to tackle the fundamental challenges of worldmodeling and establish a new paradigm for next-generationmachine reasoning . We are looking for passionate individualswho share our vision and are eager to push the boundaries of AItogether.

Key Responsibilities

• Develop the foundational world model to accurately simulate the physical world.
• Collaborate with engineering and data teams to tackle key challenges in training the world model on large-scale clusters.
• Develop metrics and evaluation benchmarks to better assess model performance.
• Design and implement a scalable and efficient data annotation pipeline to ensure high-quality labeled data for training and evaluation.
• Optimize inference efficiency to enable real-time interaction.

Areas of Focus

• Scalable Training Systems : Develop and optimize infrastructure for training multimodal LLMs and video diffusion models at massive scale.
• Efficient Data Pipelines : Build scalable video data pipelines and annotation frameworks to support high-quality training data.
• Inference Optimization : Enhance inference efficiency through optimization and distillation techniques to enable real-time interaction.
• Visual Tokenization : Develop methods for discretizing visual features into tokens for improved model representation.
• Quantitative Evaluation : Establish rigorous benchmarks to assess physical accuracy, controllability, and intelligence.
• Scaling Laws for Video Pretraining : Investigate scaling law principles to guide efficient video pre-training strategies.

Academic Qualifications

• MSc or PhD in Machine Learning or Computer Science, or equivalent industry experience.

Professional Experience

• Experience in large-scale model training (LLMs or Diffusion Models) on large clusters.
• Hands-on experience with state-of-the-art video generative models (e.g., Sora, Veo2, MovieGen, CogVideoX, etc.).
• Experiences in building and optimizing large-scale video data pipelines.
• Experience in accelerating diffusion model inference for improved efficiency.
• Exceptional problem-solving and troubleshooting skills to tackle complex technical challenges.
• Strong systems and engineering expertise in deep learning frameworks such as PyTorch.
• Strong communication and collaboration skills for effective cross-functional teamwork.
• Ability to navigate ambiguity and drive projects in rapidly evolving research areas.
• Research contributions to top-tier conferences or journals (e.g., ICML, ICLR, NeurIPS, ACL, CVPR, COLM, etc.), with published work in relevant domains.

$150,000 - $450,000 a year

Compensation

$150,000 – $450,000/year + discretionary performance bonus + benefits

The posted salary range represents the company’s good faithestimate of the base compensation for this position upon hire. Theactual base compensation offered may vary within this rangedepending on individual qualifications, including but not limitedto relevant skills, experience, education, certifications, andspecific business needs. In addition to base salary, totalcompensation includes an annual discretionary performance bonus andcomprehensive benefits.

Benefits Include

• Comprehensive medical, dental, and vision benefits
• Bonus
• 401K Plan
• Generous paid time off, sick leave and holidays
• Paid Parental Leave
• Employee Assistance Program
• Life insurance and disability

We may use artificial intelligence (AI) tools to support parts ofthe hiring process, such as reviewing applications, analyzingresumes, or assessing responses and identifying potentialinconsistencies or verification signals in application materialsbased on available information. These tools assist our recruitmentteam but do not replace human judgment. Final hiring decisions areultimately made by humans. If you would like more information abouthow your data is processed, please contact us.