Post-Doctoral Research Visit F/M Postdoc in Foundation Models for Human-Robot Interaction

Inria; Villers-lès-Nancy; Hybrid; EUR 38,000 - 48,000; Full time

Benefits offered by this job

Partial transport reimbursement

Extended leave

Teleworking after 6 months

Flexible working hours

Equipment provided

Social and cultural events

Job summary

Inria's HUCEBOT team in France invites a Post-Doctoral researcher to advance foundation models for human-robot interaction. The project focuses on grounding language into robot actions with safety for cobots, exoskeletons and humanoid platforms.

The postdoc will implement, test and develop novel algorithms for real robots, contribute to publications, and document software while collaborating with engineers and PhD students. Teleworking is possible after 6 months, with flexible hours.

Qualifications

• Significant research experience in artificial intelligence and robotics.
• Experience with robot learning, HRI, navigation or planning topics.
• Strong programming skills in Python and ML frameworks.

Responsibilities

• Implement, test and develop novel algorithms for real robots using language and foundation models.
• Write papers and present at conferences.
• Write, test, validate and document software; experiments with real robots are mandatory.

Skills

Python

PyTorch

LLM

VLM

Foundation Models

Robotics

Education

PhD in robotics/AI

Tools

PyTorch

Job description

Post-Doctoral Research Visit F/M Postdoc in Foundation Models for Human-Robot Interaction

Fonction : Post-Doctorant

The HUCEBOT team is dedicated to advancing algorithms for human-centered robots: robots that are not working autonomously in isolation, but that instead react, interact, collaborate, and assist humans. To do so, these robots need to intertwine a multi-contact whole-body controller, a digital simulation of the interacting humans, and machine learning models to predict and respond to human movements and intentions. In a crescendo of complexity, the team tackles scenarios that involve collaboration with cobots,assistance with exoskeletons, and collaboration with humanoid robots. The application domains span from industrial robotics to space teleoperation.

The main robots of the team are the mobile robot Shelfy, TiagoPro bimanual mobile manipulator, the Unitree G1 humanoid, and the Talos humanoid robot. The team also works with Franka cobots and exoskeletons.

The team currently consists of about 20+ members, including permanent researchers, PhD students and post-doctoral students.

Serena Ivaldi, head of HUCEBOT, is holding the chair in Robotics and AI of the Cluster IA ENACT project ( https://cluster-ia-enact.ai/ ) that is funding this PhD thesis. In the chair, she wants to push the research inNatural Language to assist humans in different scenarios of collaboration with robots, where safety is paramount. The ambition is to create a foundation that bridges natural language commands into interpretable commands for the robot, leading to robot actions that are contextualized and intrinsically safe.

Recent advancements in foundation models in VLMs and VLAs has drastically improved the contextual understanding and motion control for simple tasks. They allow the robot to execute plans and execute a task from visual input alone. However, there are known problems of trajectory hallucinations, safety and scaling to the long horizon problems. These problems make it difficult to make robust and reliable autonomous robot architectures. Tuning these generalized architectures and combining them with classical motion planning and control significantly improves the success rate.

When the robots have to work along with humans, strong safety guarantees are required along with clear understanding of how to interact with humans. We want to exploit the contextual understanding and action planning skills of VLMs and VLAs for motion planning in HRI with focus on safety, interaction and communication modelling and environmental adaptability.

Socially aware robot navigation across various kinds of robots and environments will be the application area of this research position.

This postdoctoral researcher will be funded by the ENACT project,but will contribute more generally to theresearch of the team in robot learning, HRI.

Main activities: implement, test and develop novel algorithms for real robots that use language models and foundation models. Write papers and present them at conferences. Write, test, validate and document its associated software. Experiments with real robots are mandatory.

This position involves contributing in the supervision of the team engineers and PhD students, together with the supervisor.

Thepostdoc will also be involved in the activities organized by the Cluster-AI project ENACT, which may involve dissemination actions, meetings and presentations to relevant stakeholders (Europe, France, industries, etc).

Good skills in Python (Pytorch). Ideally, prior experience with LLM, VLM and Foundation Models.

Goodknowledgeof robotics.

Languages: English (English is the official language of the team and many members do no speak French).

Proactivity and curiosity, daily communication, ability to work in a team are fundamental.

Avantages

• Partial reimbursement of public transport costs
• Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
• Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
• Professional equipment available (videoconferencing, loan of computer equipment, etc.)
• Social, cultural and sports events and activities

The ideal candidatehas significant research experience in artificial intelligence and robotics, robot learning, HRI, navigation, planning or analog topics. Ideal candidates have previous publications at top robotics and AI venues, such as ICRA, HRI, RSS, CORL, CVPR, ICML, Humanoids, and proven experience with real robots.