Senior Data Engineer, Data Platform - Remote

Agility Robotics; United States; Remote; USD 175,000 - 274,000; Full time

Benefits offered by this job

401(k) plan

Equity

Health insurance

Benefits start day

EAP

Flexible PTO

Holiday pay

Lunches & snacks

Parental leave

Flexible work

Professional development

Relocation assistance

Annual bonus

Job summary

Agility Robotics is seeking a Senior Data Engineer to join the Data Platform team in the United States. You will design, curate, and maintain high-quality datasets that enable analytics, debugging, and fleet-scale insights.

You will build ETL pipelines with Spark and Airflow on AWS, own gold-layer datasets, ensure data quality, and collaborate across software, hardware, and analytics teams. You may also contribute to Python/Java/C++ libraries.

Qualifications

• 5+ years of experience as a Data Engineer or similar role building and maintaining production data pipelines.
• Strong proficiency in Apache Spark or equivalent distributed data processing framework with Python, Java and/or Scala.
• Experience with Airflow, Dagster, Prefect, or other data orchestration systems.
• Proficiency with data formats such as Avro, Parquet, and structured/numeric datasets.
• Familiarity with table formats like Iceberg, Delta Lake and medallion architecture.
• Solid understanding of data modeling, schema evolution, and data quality best practices.
• Good intuitions of how to model datasets logically and partition them physically for optimal query performance.
• Nice to Have: Familiarity with Iceberg/Delta Lake, vision data pipelines, C++, data catalog systems.

Responsibilities

• Collaborate with robot software and hardware teams to define, collect, and curate data needed for analytics and debugging.
• Develop and maintain ETL pipelines that transform raw robot logs and telemetry into structured datasets using Spark, Airflow (or equivalent), and AWS data services.
• Own the gold-layer datasets that power fleet dashboards, reliability models, and business metrics with defined freshness SLAs and monitoring.
• Implement data quality checks, schema evolution, and metadata management practices.
• Collaborate with internal consumers - reliability, analytics, and ML teams - to design efficient data models for their workflows.
• Occasionally contribute to shared libraries or APIs in Python, Java, or C++ to support data capture and consumption.

Skills

Data engineering

Apache Spark

Python/Java/Scala

Airflow/Dagster

Parquet/Avro

Iceberg/Delta Lake

Data modeling

Cross-functional collaboration

Tools

Airflow

Dagster

Prefect

AWS data services

Job description

Agility Robotics is seeking a Senior Data Engineer to join the Data Platform team in the United States. You will design, curate, and maintain high-quality datasets that enable analytics, debugging, and fleet-scale insights.

You will build ETL pipelines with Spark and Airflow on AWS, own gold-layer datasets, ensure data quality, and collaborate across software, hardware, and analytics teams. You may also contribute to Python/Java/C++ libraries.