Senior 3D Spatial AI Engineer - Data & Training

NVIDIA Gruppe; Santa Clara (CA); On-site; USD 224,000 - 357,000; Full time

Job summary

NVIDIA Cosmos is hiring a Senior Deep Learning Engineer to own the data engine and end-to-end training verification behind the 3D spatial reasoning and perception capabilities of these models. You will curate and balance large-scale real-world image/video data for vision-language training, build 3D labeling pipelines, and ensure data quality and reproducibility across pre-training and fine-tuning stages.

You will collaborate with Cosmos Lab researchers to translate 3D hypotheses into dataset

Qualifications

• MS or PhD in Computer Science, Electrical/Computer Engineering, Robotics, or a related field, or equivalent experience.
• 12+ years of proven experience building deep learning systems in Python with PyTorch or JAX on Linux.
• Deep expertise in 3D computer vision, multi-view geometry, structure-from-motion or SLAM, depth and camera pose estimation, point cloud processing, or 3D reconstruction with the practical ability to produce and validate 3D ground truth at scale, not just consume it.
• Hands-on experience with vision-language models, including building the training data and evaluations that measurably improve visual grounding and reasoning quality.
• Demonstrated experience building large-scale multimodal data pipelines: distributed video and image processing, deduplication, captioning and annotation, automated quality metrics, and dataset versioning.
• Experience running and validating large model training on multi-GPU, multi-node clusters, with working knowledge of distributed training and sharding strategies such as data, tensor, and pipeline parallelism or FSDP.
• Rigorous evaluation methodology: designing benchmarks that resist gaming, building clean ablations, and reading results honestly enough to kill your own ideas.
• Excellent written and verbal communication, with a track record of partnering effectively with research scientists and translating research direction into engineering execution.

Responsibilities

• Own the 3D data engine for Cosmos spatial reasoning: source, curate, filter, and balance large-scale real-world image and video corpora into vision-language training data with the coverage and diversity that spatial understanding demands.
• Build the annotation and auto-labeling pipelines that produce 3D-grounded supervision at scale, camera-relative 3D boxes, referring and spatial question answering, free space and reachability, ego-, world-, and object-centric reference frames, cross-view correspondence, camera motion, distance and size, and chain-of-thought traces, validated by programmatic and model-based critics.
• Own data quality end to end: semantic deduplication, automated quality scoring for faithfulness, completeness, and correctness, coverage analysis across scene types and reference frames, and the sampling strategies that keep pre-training and supervised fine-tuning mixtures balanced.
• Verify end-to-end model training: run and validate full pre-training and supervised fine-tuning pipelines, guard reproducibility, catch data and checkpoint regressions, diagnose throughput and loss anomalies, and attribute capability changes back to the specific data and recipe decisions that caused them.
• Build and operate the 3D and spatial evaluation suite, public benchmarks such as CV-Bench, BLINK, RefSpatial, VSI-Bench, SPAR-Bench, and RoboSpatial, NVIDIA's VANTAGE-Bench for real-world fixed-camera video understanding, and in-house benchmarks you design with continuous evaluation and full traceability from every reported score back to the exact weights, inputs, configuration, and evaluation code.
• Partner closely with Cosmos Lab research scientists: translate 3D research hypotheses into dataset and ablation experiments, run them at scale, and feed honest results back into recipe and architecture decisions.
• Operate on large multi-node GPU clusters, tuning data throughput, sharding, and dataloader performance so that data is never the bottleneck on a long training run.
• Ship the results into Cosmos releases, open-source datasets and benchmarks where appropriate, and raise the bar for data and evaluation rigor across the team.

Skills

3D computer vision

Multi-view geometry

SLAM

Depth estimation

Point cloud processing

3D reconstruction

Vision-language models

Multimodal data pipelines

Distributed training

Evaluation methodology

Technical communication

Education

MS or PhD in CS/EE/Robotics or related field

Tools

Python

PyTorch

JAX

Linux

Job description

NVIDIA Cosmos is hiring a Senior Deep Learning Engineer to own the data engine and end-to-end training verification behind the 3D spatial reasoning and perception capabilities of these models. You will curate and balance large-scale real-world image/video data for vision-language training, build 3D labeling pipelines, and ensure data quality and reproducibility across pre-training and fine-tuning stages.

You will collaborate with Cosmos Lab researchers to translate 3D hypotheses into dataset