Senior Data Engineer, Autonomy Data Infrastructure

Tri; Los Altos (CA); On-site; USD 180,000 - 258,750; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

401(k) eligibility

Paid time off

Annual cash bonus

Job summary

Toyota Research Institute (TRI) seeks a Senior Data Engineer to design and build the foundational data infrastructure powering its autonomy research and development workflows. This includes large‑scale ingestion pipelines, feature stores, labeling infrastructure, and data discovery tools.

You will collaborate with ML researchers, autonomy engineers, and data scientists to design schemas and APIs for model training, evaluation, and debugging, while developing and maintaining data‑centric tooling

Qualifications

• 8+ years of experience building data‑intensive software systems, ideally in robotics, autonomous driving, or large‑scale ML environments.
• Proficient in Python, SQL, and familiar with C++.
• Experience designing ETL pipelines using modern frameworks (e.g., Apache Spark, Flyte, Union).
• Strong knowledge of cloud‑native architectures, including AWS services (e.g., S3).
• Familiarity with sensor data types (camera, lidar, radar, GPS/IMU) and data serialization formats (protobuf, ROS2bag, MCAP).
• Deep understanding of data quality, observability, and lineage in high‑volume systems.

Responsibilities

• Design and build scalable, production‑grade data ingestion, transformation, storage, and retrieval pipelines from vehicle fleets and simulation environments.
• Build internal tools for data labeling, curation, indexing, and cataloging across large datasets.
• Collaborate with ML researchers, autonomy engineers, and data scientists to design schemas and APIs for model training, evaluation, and debugging.
• Develop and maintain feature stores, metadata systems, and versioning for structured and unstructured data.
• Support synthetic data generation and integration with real‑world logs for hybrid training and simulation workflows.
• Optimize pipelines for cost, latency, and traceability, ensuring reproducibility across environments.
• Partner with simulation and cloud platform teams to automate workflows for closed‑loop testing, scenario mining, and analytics.

Skills

Python

SQL

C++

ETL pipelines

Spark

Cloud architectures

AWS

Sensor data

ML pipelines

Collaboration

Education

Bachelor's or Master's in Computer Science or related field

Tools

Apache Spark

Flyte

Union

SageMaker

Kubeflow

Job description

Toyota Research Institute (TRI) seeks a Senior Data Engineer to design and build the foundational data infrastructure powering its autonomy research and development workflows. This includes large‑scale ingestion pipelines, feature stores, labeling infrastructure, and data discovery tools.

You will collaborate with ML researchers, autonomy engineers, and data scientists to design schemas and APIs for model training, evaluation, and debugging, while developing and maintaining data‑centric tooling