Research Scientist - Vision Language Model

Institute of Foundation Models

Sunnyvale (CA)

On-site

USD 150,000 - 450,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

MBZUAI invites applications for a Research Scientist role in the Vision Language Model team to advance state-of-the-art multimodal foundation models. You will research architectures, data recipes, and benchmarks for large-scale VLM systems, integrating visual understanding, language reasoning, and agentive capabilities.

You will collaborate with world-class researchers and engineers, publish results, mentor junior researchers, and help scale training pipelines with reinforcement learning

Qualifications

  • PhD or equivalent research experience in Machine Learning, Computer Vision, NLP, or Multimodal AI.
  • Experience with large language models and vision-language models, including pre-training, fine-tuning, evaluation, or inference.
  • Strong Python and PyTorch development skills for large-scale ML research.
  • Experience with distributed training systems and large-scale model optimization.

Responsibilities

  • Research and development of next-generation Vision Language Models across pre-training, instruction tuning, reasoning, and agents.
  • Develop architectures and training methodologies for integrating visual understanding, language reasoning, and tool-use capabilities.
  • Research efficient multimodal learning techniques and data-efficient training.
  • Build and improve large-scale multimodal datasets, data generation pipelines, and evaluation benchmarks for VLMs.

Skills

LLM/Multimodal
Python
PyTorch
Distributed training

Education

PhD or equivalent

Tools

Megatron
Triton
CUDA
PyTorch Distributed

Job description

About the Institute of Foundation Models

We are a dedicated research lab for building, understanding, using, and risk-managing foundation models. Our mandate is to advance research, nurture the next generation of AI builders, and drive transformative contributions to a knowledge-driven economy.

As part of our team, you’ll have the opportunity to work on the core of cutting-edge foundation model training, alongside world-class researchers, data scientists, and engineers, tackling the most fundamental and impactful challenges in AI development. You will participate in the development of groundbreaking AI solutions that have the potential to reshape entire industries.Strategic and innovative problem-solving skills will be instrumental in establishing MBZUAI as a global hub forhigh-performance computing in deep learning, driving impactful discoveries that inspire the next generation of AIpioneers.


Position Summary

As a Research Scientist in the Vision Language Model (VLM) team, your role will be central to advancing state-of-the-art multimodal foundation models that integrate visual understanding, reasoning, and agentic capabilities. You will work on the research and development of large-scale VLM systems, spanning model architectures, data recipes for pre-training and post-training, and evaluation benchmarks. The role combines cutting-edge research with practical engineering, emphasizing large-scale data processing, filtering, and weighting pipelines, distributed training systems, and reinforcement learning algorithms and systems for multimodal reasoning and agent development.

  • Research and development of next-generation Vision Language Models across pre-training, instruction tuning, reasoning, and agents.

  • Develop novel architectures and training methodologies for integrating visual understanding, language reasoning, and tool-use capabilities.

  • Research efficient multimodal learning techniques, including data-efficient training, long-context modeling, model modularity, and inference optimization.

  • Build and improve large-scale multimodal datasets, synthetic data generation pipelines, and evaluation benchmarks for VLM capabilities.

  • Investigate multimodal reasoning, agentic behavior, OCR, grounding, document understanding, chart understanding, and visual question answering capabilities.

  • Contribute to technical reports, research publications, and open-source software.

  • Represent MBZUAI at research conferences and industry events, showcasing advancements in multimodal foundation models and large-scale AI systems.

  • Mentor junior researchers and collaborate across teams to drive impactful research initiatives.

PhD or equivalent research experience in Machine Learning, Computer Vision, Natural Language Processing, or Multimodal AI.

$150,000 - $450,000 a year

Salary Range

The posted salary range represents the company’s good faith estimate of the compensation for this position upon hire. The actual compensation offered may vary within this range depending on individual qualifications, including but not limited to relevant skills, experience, education, certifications, geographic location, and specific business needs.

ProfessionalExperience
Minimum
  • Experience working with large language models and/or vision-language models, including pre-training, fine-tuning, evaluation, or inference.

  • Strong Python and PyTorch development skills for large-scale machine learning research.

  • Experience with distributed training systems and large-scale model optimization.

  • Familiarity with multimodal datasets and data processing pipelines involving images, text, and video.

  • Understanding of modern deep learning architectures, including Transformers, attention mechanisms, and multimodal fusion techniques.

  • Experience with ML infrastructure, including model evaluation, debugging, optimization, and large-scale experimentation.

  • Problem-solving and research skills with the ability to independently drive research/engineering projects.

  • Effective communication and collaboration skills for working across research and engineering teams.


Preferred Skills
  • Hands-on experience training or fine-tuning large Vision Language Models or multimodal foundation models at scale.

  • Experience with distributed learning frameworks and infrastructure such as PyTorch Distributed, Megatron, Triton, or CUDA.

  • Research experience in multimodal reasoning, agentic systems, tool use, OCR, grounding, document understanding, or multimodal retrieval.

  • Experience with synthetic data generation, multimodal data curation, or automated evaluation frameworks for VLMs.

  • Familiarity with efficient training and inference techniques such as FlashAttention, quantization, tensor parallelism, pipeline parallelism, or memory optimization.

  • Experience contributing to open-source ML software and large-scale research codebases.

  • Strong publication record in leading AI conferences such as NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV, ACL, EMNLP, or related venues.

  • Experience collaborating across research, infrastructure, and product-oriented teams to deliver state-of-the‑art multimodal systems.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Research Scientist - Vision Language Model
Research Scientist - Vision Language Model

Mohamed bin Zayed University of Artificial Intelligence • Sunnyvale (CA)

On-site
USD 150,000 - 450,000
Comprehensive benefits
Bonus
401K Plan
+1
Senior Vision-Language AI Research Scientist
Senior Vision-Language AI Research Scientist

Institute of Foundation Models • Sunnyvale (CA)

On-site
USD 150,000 - 450,000
Research Scientist – World Modeling, Data
Research Scientist – World Modeling, Data

Institute of Foundation Models • Sunnyvale (CA)

On-site
USD 150,000 - 400,000
Medical benefits
Dental/vision
401K
+5
Research Scientist - NLP
Research Scientist - NLP

Institute of Foundation Models • Sunnyvale (CA)

On-site
USD 180,000 - 240,000
Comprehensive medical, dental, and vis
401K Plan
Generous paid time off
+3
Research Scientist – Multimodal Foundation Models
Research Scientist – Multimodal Foundation Models

Mecka • San Francisco (CA)

On-site
USD 185,000 - 210,000
Research Scientist - Multimodal Foundation Models
Research Scientist - Multimodal Foundation Models

Kindredventures • San Francisco (CA)

On-site
USD 180,000 - 300,000
Research Scientist - Agents (Post Training)
Research Scientist - Agents (Post Training)

Institute of Foundation Models • Sunnyvale (CA)

On-site
USD 150,000 - 450,000
Visa sponsorship
Research Scientist – Multimodal Foundation Models Toronto GTA · New York · San Francisco →
Research Scientist – Multimodal Foundation Models Toronto GTA · New York · San Francisco →

Mecka • New York (NY)

On-site
USD 140,000 - 230,000
Multimodal AI Engineer (Robotics & Foundation Models)
Multimodal AI Engineer (Robotics & Foundation Models)

Asiacruit BPO, Inc. • United States

Hybrid
USD 120,000 - 180,000
Research Scientist – Multimodal Foundation Models
Research Scientist – Multimodal Foundation Models

mecka • New York (NY)

On-site
USD 160,000 - 230,000