Data Platform Engineer, Autonomy Analytics

Medium; Irvine (CA); On-site; USD 140,000 - 180,000; Full time

Job summary

Medium is seeking a data/platform engineer in Irvine to design and build ingestion pipelines and developer tooling for Field AI. You will tackle intermittent connectivity, large sensor data, and edge-to-cloud synchronization, creating reusable SDKs, APIs, and services to onboard new robotics data sources.

You will work with streaming systems like Kafka, Kinesis, and Pub/Sub, use Airflow or Dagster for orchestration, and build integrations with BigQuery, Snowflake, Databricks, and Redshift at

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or related technical field.
• 3–5+ years of experience in data engineering or backend engineering focused on pipelines and infrastructure.
• Proficiency in Python and SQL; strong understanding of streaming systems and ETL tooling.

Responsibilities

• Design and build the data platform, frameworks, and developer tooling for ingestion across Field AI.
• Handle field data with intermittent connectivity, large sensor payloads, and edge-to-cloud sync.
• Develop reusable ingestion SDKs, APIs, and services to onboard new robotics data sources.

Skills

Python

SQL

Kafka

Kinesis

Pub/Sub

Airflow

Dagster

BigQuery

Snowflake

Databricks

Redshift

CDC/ELT tooling

Fivetran

Airbyte

Debezium

Testing

Monitoring

Teamwork

Education

Bachelor's or Master's degree in CS/Engineering

Tools

Airflow

Dagster

Fivetran

Airbyte

Debezium

Job description

What You’ll Get To Do

• Design and build the data platform, frameworks, and developer tooling that power ingestion across Field AI.
• Handle the realities of field data: intermittent connectivity, large sensor payloads (LiDAR, camera, IMU), edge-to-cloud synchronization, and backfill from offline deployments.
• Develop reusable ingestion SDKs, APIs, and services that enable teams to onboard new robotics data sources with minimal custom code.
• Build and maintain integrations across heterogeneous sources: robot/edge systems, fleet management and deployment tooling, simulation outputs, and cloud object storage.
• Integrate the platform with downstream consumers: BI tools, ML training and evaluation pipelines, labeling systems, and issue tracking.
• Develop connectors and APIs (REST/gRPC, webhooks, CDC) so internal teams can feed data in and consume curated datasets reliably.
• Own integration reliability end to end: schema contracts, versioning, retries, backfills, and monitoring.
• Optimize pipeline performance, scalability, and cost across growing fleet deployments.

What You Have

• Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field.
• 3–5+ years of experience in data engineering or backend engineering focused on pipelines and infrastructure.
• Strong programming skills in Python and SQL (C++, Scala, or Java a plus).
• Production experience with streaming systems (Kafka, Kinesis, Pub/Sub) and orchestration tools such as Airflow or Dagster.
• Experience with a modern warehouse or lakehouse (BigQuery, Snowflake, Databricks, Redshift) and cloud object storage at scale.
• Experience building integrations across systems: third-party APIs, internal services, and CDC/ELT tooling (Fivetran, Airbyte, Debezium, or custom connectors).
• Experience building for data quality: testing, monitoring, lineage, and incident response.
• Strong problem-solving skills and ability to work in interdisciplinary teams.

The Extras That Set You Apart

• Experience with robotics, autonomy, automotive, or other telemetry-heavy operational data (bag files, fleet logs, time-series sensor data).
• Familiarity with robotics middleware and log formats such as ROS/ROS2, MCAP, or rosbag.