Senior Data Platform Engineer - Multi-Source Ingest

OpenKrill; United States; Remoto; USD 140.000 - 210.000; Tempo pieno

Descrizione del lavoro

Fundable is seeking a senior data engineer to own the data platform from architecture to production. You will build a concurrent ingestion layer for providers, webhooks, files, and scraping, and orchestrate both batch and event-driven workflows using Airflow, Dagster, or Prefect.

You will preserve raw payloads in Google Cloud Storage and transform data with dbt across medallion layers. You will implement idempotent upserts, facilitate safe retries, and design robust data contracts.

Competenze

• Experience building multi-source production data platforms from architecture to operation.
• Production workflow orchestration with Airflow, Dagster, Prefect, or Temporal; Airflow preferred.
• Hands-on dbt and Google Cloud Platform experience.
• Event-driven systems with messaging or task queues and reliable retries.
• Strong SQL/PostgreSQL data modeling, loading patterns, and data contracts.
• Strong Python with asynchronous processing and concurrency.

Mansioni

• Own Fundable’s data platform architecture and production readiness.
• Ingest data from provider APIs, webhooks, files, and scraping with concurrency.
• Orchestrate batch and event-driven workflows using Airflow, Dagster, or similar.
• Preserve raw payloads in GCS and transform with dbt across medallion layers.
• Ensure data quality, validation, and monitoring of freshness and failures.

Conoscenze

SQL & PostgreSQL

Python

Data modeling

ETL pipelines

Data quality

Strumenti

Airflow

dbt

Dagster

Prefect

BigQuery

GCS

Descrizione del lavoro

Fundable is seeking a senior data engineer to own the data platform from architecture to production. You will build a concurrent ingestion layer for providers, webhooks, files, and scraping, and orchestrate both batch and event-driven workflows using Airflow, Dagster, or Prefect.

You will preserve raw payloads in Google Cloud Storage and transform data with dbt across medallion layers. You will implement idempotent upserts, facilitate safe retries, and design robust data contracts.