Multimodal Hand Pose–Language Research Intern

Honda Ri; San Jose (CA); On-site; USD 36,000 - 54,000; Full time

Job summary

Honda Research Institute USA invites a Research Intern to advance hand pose-language representation learning. You will work with egocentric video and wearables to align hand pose with language representations, enabling fine-grained cues for downstream models including LLM-based systems.

The internship focuses on developing learning methods, evaluating representations, and contributing to high-impact publications in top venues.

Qualifications

• Currently pursuing a Ph.D. in CS, EE, robotics, or related field.
• Strong ML/DL background with hands-on experience in representation learning and multimodal learning.
• Proficiency in Python and PyTorch.
• Experience with sequence modeling, egocentric video understanding, multimodal learning, or large language models.

Responsibilities

• Conduct literature reviews to identify prior work and propose directions in hand pose-language representation learning.
• Design and implement ML methods to align hand pose, egocentric video, and language using multimodal data.
• Develop hand pose encoders that learn semantic representations for multimodal reasoning and language-conditioned tasks.
• Investigate pretraining strategies aligning hand motion with natural language, akin to vision-language pretraining.
• Design rigorous experiments, evaluation protocols, and ablations to assess representation quality and task performance.
• Apply learned representations to downstream tasks such as error detection, action segmentation, and skill estimation.
• Build data processing, training, and evaluation pipelines for multimodal learning with hand pose, video, and text.
• Communicate findings through internal talks, reports, and publications.
• Collaborate with researchers to refine problems and identify publication opportunities.

Skills

Python

Machine Learning

Deep Learning

Multimodal Learning

Egocentric Video Understanding

Hand Pose Modeling

Sequence Modeling

Large Language Models

Research Communication

Education

PhD student in CS/EE/Robotics

Tools

PyTorch

Job description

Honda Research Institute USA invites a Research Intern to advance hand pose-language representation learning. You will work with egocentric video and wearables to align hand pose with language representations, enabling fine-grained cues for downstream models including LLM-based systems.

The internship focuses on developing learning methods, evaluating representations, and contributing to high-impact publications in top venues.