Data Platform Engineer — Ingestion & Automation

Minerva; New York (NY); On-site; USD 170,000 - 236,000; Full time

Job summary

Minerva is seeking a mid-level data engineer to join the data platform team in New York City. You will build ingestion pipelines for multiple sources, automate onboarding, and create reusable modules that other teams can leverage, while ensuring data lineage and privacy controls are maintained.

You will own integrations in production, handle incremental loads, backfills and failures, and work with a stack including Dagster, dbt, Snowflake, and Python.

Qualifications

• Three or more years building and supporting production data pipelines.
• Experience onboarding data from many vendors and sources, including messy data at scale.
• Practical judgment about incremental processing, backfills, failures and cost.
• Automated repetitive work or built components others engineers actually used.
• Strong Python and SQL.

Responsibilities

• Build pipelines for messy sources (APIs, files, DB extracts, web scrapes).
• Make reruns safe with incremental loads, backfills, retries, and clean data quality.
• Automate onboarding from source config to first delivery.
• Build reusable data ingestion modules with tests and clear interfaces.
• Preserve source origin metadata and enforce privacy controls.
• Own production integrations with monitoring and recovery procedures.

Skills

Python

SQL

Dagster

Airflow

dbt

SQLMesh

Snowflake

Spark

Iceberg

Trino

AWS Glue

Tools

Dagster

Airflow

dbt

SQLMesh

Snowflake

Spark

Iceberg

Trino

AWS Glue

Job description

Minerva is seeking a mid-level data engineer to join the data platform team in New York City. You will build ingestion pipelines for multiple sources, automate onboarding, and create reusable modules that other teams can leverage, while ensuring data lineage and privacy controls are maintained.

You will own integrations in production, handle incremental loads, backfills and failures, and work with a stack including Dagster, dbt, Snowflake, and Python.