Research Scientist Intern, Multimodal Learning for Robotics (PhD)

Meta; Burlingame (CA); On-site; USD 44,000 - 66,000; Full time

Job summary

Reality Labs Research at Meta seeks a Research Scientist Intern to advance CV and tactile-based algorithms for robotic control in dexterous manipulation tasks. You will work at the intersection of computer vision, ML, and human-robot interaction.

Ideal candidates hold a PhD in CS/AI/Robotics and have hands-on experience with PyTorch, Python, and Mujoco, plus exposure to multimodal learning and foundation CV models. This internship spans 12–24 weeks with multiple start dates.

Qualifications

• Ph.D. in Computer Science, AI, Robotics or relevant field.
• Experience in computer vision with active contributions in subdomains (3D reconstruction, hand pose estimation, object pose estimation).
• Understanding state of the art in robotic manipulation control; experience with physics-based simulators like Mujoco.

Responsibilities

• Conduct collaborative research on CV and tactile-based algorithms for human and robotics data.
• Develop robotic data pipelines, train models, and benchmark algorithms.
• Collaborate with researchers and cross-functional teams, communicating research plans and results.

Skills

Computer vision

Python

PyTorch

JAX

Robotics

Mujoco

Multimodal learning

Foundation CV models

Cross-functional communication

Education

Ph.D. in Computer Science/AI/Robotics

Tools

Mujoco

CLIP

DinoV2

SAM2

Job description

About

At Reality Labs Research, our goal is to explore, innovate and design novel interfaces and hardware for the next generation of virtual, augmented, and mixed reality experiences. We are driving research towards a vision of natural, seamless experiences in XR environments that are effective, enjoyable, and functionally indistinguishable from those in the real world.As a Research Scientist Intern, you will work at the intersection of robotic control, machine learning, and human-robot interaction, and you will have the opportunity to work with world-leading collaborators and mentors in these fields. Your primary focus will be on developing CV and tactile based algorithms that can be used for robotic control learning in dextrous manipulation tasks.Our internships are twelve (12) to twenty-four (24) weeks long and we have various start dates throughout the year. Some projects may require a minimum of 16 consecutive weeks.

Responsibilities

• Conduct collaborative research on developing CV and tactile based algorithms for both human and robotics data.
• Develop robotic data pipelines for robotic dextrous manipulation, train models, and benchmark different algorithms.
• Collaborate with researchers and cross-functional partners including communicating research plans, progress, and results.

Minimum Qualifications

• Currently has or is in the process of obtaining a Ph.D. degree in Computer Science, Artificial Intelligence, Robotics, or relevant technical field
• Experience in computer vision, with active contributions in at least one subdomain (3D reconstruction, hand pose estimation, object pose estimation) and strong familiarity across all
• Understand state of the art in robotic manipulation control policies, with experience in physics based simulators like Mujoco
• Experience with Python, PyTorch or JAX, or other related languages
• Must obtain work authorization in the country of employment at the time of hire and maintain ongoing work authorization during employment Proven track record of achieving significant results as demonstrated by grants, fellowships, patents, as well as first-authored publications at leading workshops or conferences such as Robotics (RSS, ICRA, IROS, CoRL, T-RO, IJRR), Machine Learning (NeurIPS, ICML, ICLR, AAAI, JMLR), and Computer Vision (CVPR, ICCV, ECCV, TPAMI), or similar
• Experience with multimodal learning including but not limited to tactile data
• Experience working with robot manipulation
• Experience with building pipelines with foundation CV models, for example, CLIP, DinoV2 or SAM2
• Experience working and communicating cross functionally in a team environment
• Intent to return to the degree program after the completion of the internship/co-op
• Availability for minimum 16 consecutive week internship