Research engineer Perspective-Grounded Spatial Reasoning in Multimodal Large Language Models

Inria Paris; Paris; On-site; EUR 26,000 - 35,000; Full time

Job summary

Inria Paris invites applications for a research-oriented position focusing on multimodal large language models with an emphasis on spatial reasoning and perspective-taking. You will align models with 3D geometric structures using pretrained foundations models and contribute to diagnostic benchmarks.

The role involves analyzing related work, designing novel methods, and communicating progress through reports and papers, with opportunities to present at conferences and publish findings.

Qualifications

• Programming skills with practical experience.
• Proficiency in English; French is a plus.
• Good written and verbal communication.

Responsibilities

• Analyse and implement related work.
• Design novel innovative solutions.
• Write progress reports and papers.
• Present work at conferences.

Skills

Programming

English

French (optional)

Communication

Education

PhD/ MSc in CS

Tools

PyTorch

TensorFlow

3D foundation models

Job description

Assignment Multimodal large language models (MLLMs) remain notably weak at spatial reasoning from video, particularly at perspective-taking where the model is asked to answer questions from a viewpoint other than the camera's own (e.g., identifying what lies to an observer's left from an allocentric frame). Existing approaches largely rely on either naive data augmentation or opaque latent reasoning tokens whose targets are not grounded in real geometry, both of which have known failure modes. This project aims to build a diagnostic benchmark that isolates genuine perspective-taking failures from confounds (e.g., left-right mirroring, temporal-order artifacts), and to develop a spatial reasoning representation that is explicitly grounded in 3D geometric structure via alignment with a pretrained 3D foundation model, rather than relying on self-consistency alone.

Main activities

• Analyse and implement related work.
• Design novel innovative solutions.
• Write progress reports and papers.
• Present work at conferences.

Skills

Technical skills and level required : programming skills are required. Languages : English and possibly French. Relational skills : Good communication skills.