Robot Learning Engineer

Lightspeed; Northbrook (IL); On-site; USD 100,000 - 120,000; Full time

Job summary

LightSpeed Build Technologies is seeking a Robot Learning Engineer to teach our robots complex manipulation tasks, from demonstration to deployed behavior. You will own the path from demonstration to deployed behavior, building teleoperation systems, turning data into trained policies, and proving those policies on real robots with dexterous hands and custom tooling.

You will design data collection protocols, retarget human motion, and shape data strategies while guiding technicians who run

Qualifications

• 3+ years hands-on robot learning or production ML experience.
• Experience training policies from demonstrations on physical robots.
• Experience collecting teleoperation or demonstration data.
• Strong PyTorch skills for end-to-end training, evaluation, debugging.

Responsibilities

• Build and improve teleoperation pipelines using leader arms and VR hand tracking.
• Retarget human hand motion to a dexterous robot hand, in simulation then on hardware.
• Design data collection protocols for consistent, trainable demonstrations.
• Define data worth and guide technicians during collection sessions.
• Train and fine-tune manipulation policies using imitation learning and VLA models.
• Choose policy architectures for multi-step, contact-rich tasks.
• Combine teleoperation data with video to improve policies.
• Iterate from data to trained policy to real-robot trial.
• Define evaluation protocols and measure task success on real robots.
• Diagnose failures across data, model, and hardware; fix root causes.

Skills

Robot learning

Imitation learning

PyTorch

Python

C++

ROS/ROS2

Linux

Git

Education

MS or PhD in Robotics or ML

Tools

MuJoCo

Isaac Sim

Docker

rosbag / MCAP

Job description

About LightSpeed

LightSpeed Build Technologies is revolutionizing the construction industry through AI-powered robotics. Our flagship systems—BRUTE for automated wall panel manufacturing and DEX for on-site collaborative construction—are addressing the global housing crisis by delivering unprecedented speed, precision, and affordability in homebuilding.

Position Overview

As a Robot Learning Engineer, you will teach our robots to perform complex manipulation tasks from human demonstration. You will own the path from demonstration to deployed behavior: building the teleoperation systems that capture how skilled people do the work, turning that data into trained policies, and proving those policies on real robots with dexterous hands and custom tooling. You'll also shape how we learn from video of people doing the work itself. This role is hands‑on, fast‑moving, and measured by one thing: robots that reliably do the job.

What You'll Work On

Teleoperation & Demonstration Data

• Build and improve teleoperation pipelines using leader arms for bulk arm motion and VR hand tracking for fine, finger-level interaction
• Retarget human hand motion to a dexterous robot hand, developing in simulation first and validating on hardware
• Design data collection protocols so demonstrations are consistent, well‑structured, and train well
• Define what data is worth collecting and how much, and guide technicians who run collection sessions

Policy Learning

• Train and fine‑tune manipulation policies using imitation learning and vision‑language‑action (VLA) models
• Choose policy architectures and action representations suited to multi‑step, contact‑rich tasks
• Combine teleoperation data with human demonstration video to improve policies
• Iterate quickly from data to trained policy to real‑robot trial

Evaluation & Deployment

• Define and run evaluation protocols on real robots, measuring success rate per task step
• Diagnose failures across data, model, and hardware, and fix the right one
• Deploy policies on robots with ROS2‑based control and optimize inference for real‑time performance
• Work with integration engineers to run policies across a full task sequence

Data Pipeline & Vendor Oversight

• Define dataset formats and data quality standards for training
• Supervise external partners who build data pipeline infrastructure and label human demonstration video, and set specs and accept deliveries
• Prototype extraction of useful training signals from egocentric (head‑mounted camera) video in‑house before scaling it out

Required Qualifications

Robot Learning

• 3+ years of hands‑on experience in robot learning, ML for robotics, or closely related production ML
• Experience training policies from demonstrations on physical robots using imitation learning (e.g., ACT, Diffusion Policy, or similar)
• Experience collecting teleoperation or demonstration data for robot learning
• Strong PyTorch skills and experience running training, evaluation, and debugging end to end
• Track record of evaluating models in the real world and diagnosing failures

Robotics & Software

• Strong Python; working C++
• Experience with ROS or ROS2 and real robot hardware
• Understanding of manipulator kinematics and control fundamentals
• Proficiency with Linux, Docker, and Git

Preferred Qualifications

• MS or PhD in Robotics, Machine Learning, Computer Science, or a related field
• Experience fine‑tuning VLA or foundation models for robot control
• Experience with dexterous hands, VR hand tracking, or motion retargeting
• Experience building or using leader‑follower teleoperation systems
• Experience learning from human video or egocentric data
• Experience building and maintaining large‑scale robotics datasets
• Robotics simulation experience with MuJoCo, Isaac Sim, or similar
• Experience with robot learning datasets and tooling (e.g., LeRobot, rosbag, MCAP)
• Experience managing data vendors or labeling workflows
• Background in manufacturing, industrial automation, or construction technology

Why Join LightSpeed

• Teach real robots to do skilled work that addresses the housing crisis
• Own the full loop from human demonstration to deployed robot behavior
• Work with dexterous hands, custom tooling, and human demonstration data at scale
• Hands‑on culture with direct access to robots, sensors, and manufacturing environments
• Competitive compensation and comprehensive benefits

Employment Relationship

This position is at‑will, meaning either you or the Company may terminate employment at any time, with or without cause or notice.

Equal Opportunity

LightSpeed Build Technologies is an equal opportunity employer committed to building a diverse and inclusive workplace. We welcome candidates from all backgrounds and experiences.

Compensation Range: $100K - $120K