Senior Data Platform Engineer: Flight Data at Scale

Neros Technologies; Torrance (CA); On-site; USD 164,000 - 229,000; Full time

Job summary

Neros Technologies seeks a Data Platform Engineer to own the end-to-end autonomy data pipeline. You will recover flight data from the field, land it in the cloud, and make it searchable and replayable at scale.

This greenfield role includes storage, catalog, and query infrastructure ownership for all autonomy tests, simulations, and ML datasets. You will design the catalog, stand up storage and databases, and build a query layer for rapid retrieval and replay of flight data, with reproducible

Qualifications

• 5+ years building production data or backend infrastructure.
• Direct experience with large-scale log or sensor data: multi-terabyte and growing, with video and multiple synchronized sensor streams.
• Designed and owned a data schema, index, or catalog that other engineers queried daily, and lived with the consequences of that design, including at least one migration.
• Strong Python, plus SQL and working ownership of a relational database (PostgreSQL or equivalent) used in production.
• Practical experience with cloud object storage and compute (Azure, AWS, or GCP) and the ability to provision and operate it independently, without a dedicated platform or DevOps team.
• Experience with distributed or parallel batch processing and job orchestration (Spark, Ray, Dask, Airflow, Dagster, or equivalent).
• Working knowledge of time synchronization across sensor streams and deterministic, reproducible processing of recorded data.
• Track record of building internal tooling that other engineers adopted.

Responsibilities

• Build the data pipeline to recover flight data from the field, ensure prioritized uploads and resumable transfers with integrity checks.
• Turn raw logs into a usable corpus by decoding, time-aligning, validating, and quarantining malformed data.
• Design the catalog, index, and cloud storage/database for the data corpus.
• Build the query layer so engineers can retrieve flights matching conditions and replay video quickly.
• Provide logs to the evaluation harness with stable ordering and reproducible results across runs.
• Create versioned, immutable datasets with lineage for reproducible model training.

Skills

Python

SQL

PostgreSQL

Cloud platforms

Data pipeline design

Time synchronization

Tooling adoption

End-to-end ownership

Tools

Spark

Ray

Dask

Airflow

Dagster

MCAP/ROSbag/HDF5/Parquet

Job description

Neros Technologies seeks a Data Platform Engineer to own the end-to-end autonomy data pipeline. You will recover flight data from the field, land it in the cloud, and make it searchable and replayable at scale.

This greenfield role includes storage, catalog, and query infrastructure ownership for all autonomy tests, simulations, and ML datasets. You will design the catalog, stand up storage and databases, and build a query layer for rapid retrieval and replay of flight data, with reproducible