Research Intern: Multimodal Action and Skill Understanding in Procedural Activities

Honda Research Institute USA; San Jose (CA); On-site; USD 33,000 - 47,000; Full time

Job summary

Honda Research Institute USA in San Jose, CA invites a research intern to advance multimodal action understanding in long procedural activities. You will fuse video, pose, force/tactile signals, and other modalities to estimate proficiency and segment actions.

The role offers collaboration on publications and potential patents. The internship requires an MS/PhD in robotics, CV, ML, or AI, strong PyTorch skills, and the ability to work independently and communicate clearly.

Qualifications

• Currently enrolled in an M.S. or Ph.D. program in Robotics, Computer Vision, Machine Learning, Artificial Intelligence, or a closely related field.
• Experience with video understanding and temporal modeling, including modern video encoders or multimodal foundation models.
• Experience combining video with at least one non-visual modality (pose, IMU, force/tactile, audio).
• Strong programming skills and proficiency in PyTorch, with the ability to write clean, reproducible research code.
• Ability to independently drive a project from ideation to experimentation (and optionally publication).
• Strong written and verbal communication skills.

Responsibilities

• Design and implement multimodal methods for temporal action segmentation and proficiency estimation in long, untrimmed procedural recordings, combining multiview video with pose, force, and/or tactile signals.
• Develop approaches for learning across modalities to improve vision-only inference.
• Build skill representations that connect what a person does with how well they do it, and study how they transfer across tasks and viewpoints and sensor configurations.
• Benchmark rigorously, conduct error analysis, and build tooling for reproducible experimentation.
• Collaborate with the team and contribute to publications (e.g., CVPR, ICCV, ECCV, NeurIPS) and patents.

Skills

Video understanding

Temporal modeling

PyTorch

Research coding

Independent project drive

Communication

Multimodal fusion

Education

M.S. or Ph.D. in Robotics/Computer Vision/ML/AI or related field

Tools

PyTorch

Job description

Research Intern: Multimodal Action and Skill Understanding in Procedural Activities - Honda Research Institute USA

Research Intern: Multimodal Action and Skill Understanding in Procedural Activities

Job Number: P26INT-09

Honda Research Institute USA (HRI-US) is seeking a highly motivated and independent research intern to join our team working on human action and skill understanding in long procedural activities. This internship builds on our research in temporal action segmentation and proficiency estimation, leveraging multi-view videos along with hand and body pose, force, and tactile signals. Potential research topics include, but are not limited to, multimodal fusion for segmentation and skill assessment, as well as skill representations grounded in motion and interaction dynamics. Depending on research outcomes, the intern may contribute to high-impact publications and patents.

San Jose, CA

Key Responsibilities

• Design and implement multimodal methods for temporal action segmentation and proficiency estimation in long, untrimmed procedural recordings, combining multiview video with pose, force, and/or tactile signals.
• Develop approaches for learning across modalities to improve vision-only inference.
• Build skill representations that connect what a person does with how well they do it, and study how they transfer across tasks and viewpoints and sensor configurations.
• Benchmark rigorously, conduct error analysis, and build tooling for reproducible experimentation.
• Collaborate with the team and contribute to publications (e.g., CVPR, ICCV, ECCV, NeurIPS) and patents.

Minimum Qualifications

• Currently enrolled in an M.S. or Ph.D. program in Robotics, Computer Vision, Machine Learning, Artificial Intelligence, or a closely related field.
• Experience with video understanding and temporal modeling, including modern video encoders or multimodal foundation models.
• Experience combining video with at least one non-visual modality (e.g., pose, IMU, force/tactile, audio).
• Strong programming skills and proficiency in PyTorch, with the ability to write clean, reproducible research code.
• Ability to independently drive a project from ideation to experimentation (and optionally publication).
• Strong written and verbal communication skills.

Bonus Qualifications

• Publications at top-tier venues (e.g., CVPR, ICCV, ECCV, NeurIPS, ICRA, IROS).
• Experience with temporal action segmentation, skill assessment, or error detection in procedural videos.
• Experience with force/tactile sensing, contact estimation, or human-object interaction.
• Experience with multi-view or cross-view learning (e.g., egocentric-exocentric).
• Familiarity with generative models.
• Familiarity with self-supervised representation learning or learning with missing modalities.

Years of Work Experience Required 0

Desired Start Date May 2027

Internship Duration 3 Months

Position Keywords Video Action Segmentation, Human Action Understanding,
Proficiency Estimation, Human Object Interaction, Robotics

For California applicants: Please review the Honda California Job Applicant Privacy Notice before submitting your application.