Research Engineer — VR, Multimodal HRI

Singapore Management University; Singapore; On-site; SGD 60,000 - 90,000; Full time

Job summary

Singapore Management University invites applications for a research-focused role in the School of Computing and Information Systems. The successful candidate will extend a VR testbed using Unity (front-end) and Python (back-end), integrating live speech, eye, and hand tracking.

They will design a multimodal reference resolver with <200 ms latency and create visual overlays illustrating the agent’s belief. The role includes running controlled lab studies (N≈24–48), IRB tasks, and preparing

Qualifications

• Master’s degree in Computer Science, Information Technology, Information Systems or closely related disciplines with focus on Human-Robot Interaction.
• Minimum 1 year of relevant research experience in Human-Robot Interaction as graduate student, research assistant or research engineer.
• Proficiency in programming languages such as C# and Python, and the Unity environment is required.
• Hands-on experience developing for VR head-mounted displays (e.g., Meta Quest, OpenXR).
• Experience integrating real-time multimodal input pipelines (speech recognition, gaze, gesture) and/or LLM/VLM APIs into interactive systems.
• Proven track record in interface prototyping, user experimental design and academic manuscript preparation.

Responsibilities

• Extend and maintain the lab's VR testbed (Unity front-end, Python back-end) with live speech recognition, eye tracking, hand tracking, and scene-graph representation.
• Implement an incremental multimodal reference resolver fusing speech, gaze, and gesture with real-time latency <200ms.
• Design and implement in-situ visual overlays showing the agent’s belief and integrate with the Co-PI's communication policy.
• Design, pilot, and run controlled lab user studies (N≈24–48), including scripting, instrumentation, recruitment and IRB documentation.
• Analyze quantitative and qualitative study data and co-author submissions to CHI, UIST, DIS, CSCW, or HRI.
• Collect, annotate, and release a public corpus of natural deictic instructions and maintain annotation scheme.

Skills

C#

Python

Human-Robot Interaction

Education

Master’s degree in Computer Science / IT / IS with Human-Robot Interaction focus

Tools

Unity

OpenXR

Job description

Singapore Management University invites applications for a research-focused role in the School of Computing and Information Systems. The successful candidate will extend a VR testbed using Unity (front-end) and Python (back-end), integrating live speech, eye, and hand tracking.

They will design a multimodal reference resolver with <200 ms latency and create visual overlays illustrating the agent’s belief. The role includes running controlled lab studies (N≈24–48), IRB tasks, and preparing