VLM Engineer

Technology Innovation Institute; United Arab Emirates; On-site; AED 400,000 - 700,000; Full time

Job summary

Technology Innovation Institute (TII) in Abu Dhabi leads pioneering AI research within the Artificial Intelligence and Digital Research Centre, focusing on extreme-scale language and vision-language models. This role supports cutting-edge research, collaborating with international partners to push state-of-the-art in AI, cryptography, and related fields.

The successful candidate will conduct ablation studies, curate datasets for vision-language tasks, develop evaluation protocols, and contribute

Qualifications

• PhD in deep learning required.
• Proven track record in vision-language models.
• Publication record in top-tier conferences desirable.

Responsibilities

• Conduct comprehensive ablation studies on vision models and report findings.
• Partner with researchers to analyze data ablation studies for training vision-language models.
• Develop and implement robust evaluation protocols for vision-language models across benchmarks and real-world scenarios.
• Engage in model training, integrating large language models with vision models and optimizing multi-GPU training pipelines.

Skills

Vision-language models

CLIP architectures

PyTorch

Distributed training

Model evaluation

Data curation

Education

PhD in deep learning

Tools

PyTorch

Job description

Technology Innovation Institute (TII) is a publicly funded research institute, based in Abu Dhabi, United Arab Emirates. It is home to a diverse community of leading scientists, engineers, mathematicians, and researchers from across the globe, transforming problems and roadblocks into pioneering research and technology prototypes that help move society ahead.

Artificial Intelligence and Digital Research Centre

As part of TII’s Artificial Intelligence Research Center, the Extreme‑Scale Language Model team is developing and implementing innovative deep learning technologies with a broad range of applications, from Natural Language Processing to Perception and Vision. Our team has developed the Falcon models and is now planning to continue our journey into cutting‑edge applied research in the topic of large language models.

Key Responsibilities

• Conduct comprehensive ablation studies on vision models to assess the impact of various components and configurations, and collaborate with researchers to analyze and report on the effectiveness of different model architectures and settings.
• Partner with team members to perform data ablation studies, identifying optimal data types and structures for training vision‑language models, and analyze the impact of different data inputs on model performance, with a focus on vision‑language alignment.
• Develop and implement robust evaluation protocols for vision‑language models and assess model performance across diverse benchmarks and real‑world scenarios.
• Engage in model training, with an emphasis on integrating large language models with vision models like CLIP, and optimize multi‑GPU training pipelines.

Skills Required

• Expertise in machine learning, particularly in vision‑language models and large language models.
• Strong understanding of model architectures such as CLIP and their application in vision‑language tasks.
• Proficiency in distributed training techniques and multi‑GPU optimization.
• Experience with deep learning frameworks (e.g., PyTorch).
• Strong analytical skills for conducting ablation studies and evaluating model performance.
• Familiarity with dataset curation and processing for vision and language tasks.

Qualifications

• PhD degree in deep learning.
• Proven track record of research and development in vision‑language models.
• Publication record in top‑tier conferences is highly desirable.

At TII, we help society to overcome its biggest hurdles through a rigorous approach to scientific discovery and inquiry, using state‑of‑the‑art facilities and collaboration with leading international institutions. Our rigorous discovery and inquiry‑based approach helps to forge new and disruptive breakthroughs in AI, advanced materials, autonomous robotics, cryptography, digital security, directed energy, quantum computing and secure systems.