Research Scientist Intern, Low-Power Scene Reconstruction for AR/VR

Meta; Redmond, Burlingame (WA, CA); On-site; USD 60,000 - 80,000; Full time

Job summary

Meta Reality Labs is seeking a Research Scientist Intern to tackle low-power real-time reconstruction of scene geometry for AR/VR. You will define approaches for depth estimation under glasses power budgets and evaluate lightweight representations that support passthrough, occlusions, and background scenes.

The role involves owning an open problem from method selection to evaluation, with opportunities for publication where the work merits it, and collaboration with the XR Graphics team.

Qualifications

• Must be pursuing PhD or equivalent degree in CV/ML/graphics/robotics
• Experience designing and running quantitative experiments and communicating results
• Experience with on-device inference: quantization, pruning, distillation, or deployment to mobile targets
• Experience with multi-sensor fusion across heterogeneous sensors (RGB, IR, ToF) including calibration
• Familiarity with AR/VR passthrough, reprojection, occlusion rendering or SLAM

Responsibilities

• Own an open problem in depth estimation for low-power AR/VR systems
• Design and run quantitative experiments and build a evaluation framework
• Present results and contribute to publications where appropriate
• Collaborate with XR Graphics and Reality Labs teams to define approach and share findings

Education

PhD in computer vision, machine learning, computer graphics, robotics, or related field

Tools

Profiling tools

Unix environment

C++

Job description

Future AR and VR devices occupy ever-smaller, glasses-like form factors and thus require ever-tighter power budgets. At the same time, we require higher-quality passthrough rendering and scene understanding. In particular, depth estimation, which reconstructs scene geometry in real time from sensor signals, is one of the biggest power consumers in AR and VR devices. However, most existing work targets datacenter GPUs, or at best mobile silicon with an order of magnitude more power headroom than what glasses form factors allow.

Reality Labs is Meta's innovation engine for next-generation AR/VR and wearable technology and XR Graphics is involved in building the next-generation imaging and reconstruction stack that closes this gap.

We are looking for a Research Scientist Intern to work on low-power real-time reconstruction of scene geometry for AR/VR.

• What is the right depth estimation and processing strategy, from the ground up, for a glasses power budget?
• Can we train a single lightweight scene representation that can serve passthrough, hand occlusions, and background occlusions simultaneously?

These are open questions, and you would help define the approach. This is a research internship. You will own a genuinely open problem, choose the methods, build the evaluation, and present your results, with support for publication where the work warrants it.

• Currently has, or is in the process of obtaining, a PhD degree in the domain of computer vision, machine learning, computer graphics, robotics, or a related technical field
• Must obtain work authorization in the country of employment at the time of hire, and maintain ongoing work authorization during employment
• Knowledge and hands-on experience in at least one of: depth or stereo estimation, 3D reconstruction (TSDF, implicit or neural fields, Gaussian splatting), video or instance segmentation, or neural scene representations
• Experience designing and running quantitative experiments, and communicating technical results
• Intent to return to a degree-program after the completion of the internship
• Proven track record of achieving significant results as demonstrated by grants, fellowships, patents, as well as first-authored publications at leading workshops or conferences such as CVPR/ECCV/ICCV, SIGGRAPH/SIGGRAPH Asia, 3DV, ICLR, NeurIPS, etc
• Experience with efficient or on-device inference: quantization, distillation, pruning, or deployment to mobile NPU, DSP or GPU targets
• Experience with multi-sensor fusion across heterogeneous sensors — RGB, IR/CV cameras, time-of-flight — including camera calibration and cross-sensor registration
• Experience with temporal filtering and state estimation over scene state
• Familiarity with AR/VR passthrough, reprojection, occlusion rendering, or SLAM and scene reconstruction
• Experience with C++, and with profiling or optimizing real-time computer vision or graphics pipelines
• Experience working in a Unix environment
• Experience reasoning about compute and power budgets as a first-class design constraint