Senior Data Engineer

Gatik AI

Santa Clara (CA)

On-site

USD 180,000 - 260,000

Full time

5 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Gatik AI in Santa Clara, CA is seeking a senior data infrastructure engineer to build and own production pipelines that transform fleet and simulation telemetry into training-ready datasets for driverless operations.

You will design end-to-end data ingestion, labeling, indexing, and cataloging tools, collaborate with ML researchers, and develop feature stores and data lineage for high-volume sensor data across camera, lidar, radar, GPS/IMU.

Qualifications

  • 8+ years building data-intensive software systems.
  • Strong proficiency in Python and SQL with working familiarity in C++.
  • Hands-on experience designing ETL/ELT pipelines with Spark, Flyte, Union, Ray or Airflow.
  • Strong knowledge of cloud architectures on AWS or Google Cloud.
  • Familiarity with sensor data formats Protobuf, ROS/ROS 2 bags, MCAP.

Responsibilities

  • Design, build, and operate production-grade data pipelines ingesting multi-sensor logs from autonomous fleet and simulations.
  • Create internal tools for data labeling, curation, indexing, and cataloging to locate drives, events, and scenarios quickly.
  • Collaborate with ML researchers and engineers to define schemas and APIs for model training and evaluation.
  • Develop feature stores, metadata systems, and dataset versioning for structured and unstructured data.
  • Integrate synthetic data with real logs to support hybrid training and simulation workflows.
  • Surface rare safety-relevant events via scenario-mining and event-tagging pipelines.

Skills

Python
SQL
C++
ETL/ELT pipelines
Apache Spark
Flyte
Union
Ray
Airflow
AWS
Google Cloud
Protobuf
ROS/ROS 2 bags
MCAP

Education

Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field

Tools

SQL
Python
C++
Apache Spark
Flyte
Union
Ray
Airflow
AWS
Google Cloud
Protobuf
ROS/ROS 2 bags
MCAP

Job description

Build and own production data infrastructure that converts fleet and simulation telemetry into training-ready datasets for driverless operations.

Responsibilities
  • Design, build, and operate production-grade pipelines to ingest, decode, transform, and store multi-sensor logs from Gatik’s autonomous truck fleet and simulation environments.
  • Create internal tools and services for data labeling, curation, indexing, and cataloging to help engineers locate the right drives, events, and scenarios quickly.
  • Collaborate with ML researchers, autonomy engineers, and data scientists to define schemas and APIs for model training, evaluation, and debugging.
  • Develop and maintain feature stores, metadata systems, and dataset versioning for structured data (events, metrics, vehicle signals) and unstructured data (camera, lidar, radar).
  • Integrate synthetic and simulated data with real-world logs to support hybrid training, scenario replay, and simulation workflows.
  • Build scenario-mining and event-tagging pipelines to surface rare and safety-relevant situations on routes, including unprotected turns, dock approaches, and adverse weather.
  • Optimize for cost, latency, and traceability, including end-to-end lineage from raw logs to trained model and evaluation results.
  • Partner with simulation, cloud platform, and safety teams to automate closed-loop testing, regression analysis, and fleet performance analytics.
  • Set data quality and observability standards and mentor engineers through design reviews and code reviews.
Requirements
  • Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field, or equivalent practical experience.
  • 8+ years building data-intensive software systems, ideally in autonomous driving, robotics, or large-scale ML environments.
  • Strong proficiency in Python and SQL, with working familiarity with C++.
  • Hands-on experience designing ETL/ELT pipelines with modern frameworks such as Apache Spark, Flyte, Union, Ray, or Airflow.
  • Strong knowledge of cloud-native architectures on AWS (S3, EC2, EKS, Glue, Athena) or Google Cloud equivalents.
  • Familiarity with sensor data (camera, lidar, radar, GPS/IMU) and serialization formats such as Protobuf, ROS/ROS 2 bags, and MCAP.
  • Deep understanding of data quality, observability, and lineage in high-volume systems.
  • Proven ability to deliver reliable, performant infrastructure that supports both ad-hoc exploration and repeatable production workflows.
Technologies
  • Python, SQL, C++, Apache Spark, Flyte, Union, Ray, Airflow
  • AWS (S3, EC2, EKS, Glue, Athena), Google Cloud
  • Protobuf, ROS/ROS 2 bags, MCAP
Nice to have
  • Experience in AD/ADAS, autonomous trucking, or robotics, especially with perception, prediction, or planning datasets.
  • Familiarity with ML orchestration platforms such as Kubeflow, SageMaker, or Vertex AI.
  • Experience with temporal and spatial data, including geospatial indexing (H3, S2, PostGIS), HD maps, and multi-sensor time-series alignment.
  • Exposure to synthetic data generation, simulation logging, or scenario replay pipelines.
  • Experience with lakehouse and table formats such as Delta Lake, Apache Iceberg, or Parquet at petabyte scale.
  • Familiarity with safety-critical development practices and data traceability for safety cases or regulatory requirements.
  • Strong software engineering fundamentals: CI/CD, testing, code review, and service deployment best practices.
  • Experience working with cross-functional, distributed teams across research and production organizations.
Location and Salary
  • Location: Santa Clara, CA (onsite)
  • Salary: USD 180,000 - 260,000 per year

By submitting your application, you acknowledge that Gatik participates in the U.S. Department of Homeland Security E-Verify program.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer Santa Clara, CA
Senior Data Engineer Santa Clara, CA

Gatik • Santa Clara (CA)

On-site
USD 180,000 - 260,000
Senior Data Engineer
Senior Data Engineer

Gatik • Santa Clara (CA)

On-site
USD 180,000 - 260,000
Senior Data Engineer: Build Autonomous Driving Platform
Senior Data Engineer: Build Autonomous Driving Platform

Gatik • Santa Clara (CA)

On-site
USD 180,000 - 260,000
Senior Software Engineer, ML Infrastructure
Senior Software Engineer, ML Infrastructure

Gatik • California

On-site
USD 180,000 - 260,000
Technical Lead, ML Infrastructure Santa Clara, CA
Technical Lead, ML Infrastructure Santa Clara, CA

Gatik • Santa Clara (CA)

On-site
USD 180,000 - 240,000
Technical Lead, ML Infrastructure
Technical Lead, ML Infrastructure

Gatik • Santa Clara (CA)

On-site
USD 180,000 - 240,000
Senior Data Engineer: Autonomous Driving Data Platform
Senior Data Engineer: Autonomous Driving Data Platform

Gatik • Santa Clara (CA)

On-site
USD 180,000 - 260,000
Senior Software Engineer, ML Infrastructure Santa Clara, CA
Senior Software Engineer, ML Infrastructure Santa Clara, CA

Gatik • Santa Clara (CA)

On-site
USD 180,000 - 260,000
Senior Software Engineer, ML Infrastructure
Senior Software Engineer, ML Infrastructure

Gatik • Santa Clara (CA)

On-site
USD 180,000 - 260,000
Senior Software Engineer, C++
Senior Software Engineer, C++

Gatik • Santa Clara (CA)

On-site
USD 180,000 - 260,000