Senior Data Platform Engineer: Ingest, Orchestrate & Scale

silver; United States; Remote; USD 140,000 - 180,000; Full time

Job summary

silver seeks a data platform engineer to own the architecture, implementation, and production operation of a data platform consolidating data from APIs, webhooks, files, and web scraping. You will evolve serial workflows into concurrent, event-driven intake and orchestrated batch processing, delivering traceable records to downstream products.

You will build ingestion across providers, orchestrate jobs with a workflow engine, preserve source payloads in cloud storage, and implement robust,

Qualifications

• Experience building production data platforms from architecture to deployment.
• Proficiency with workflow orchestration tools and retries.
• Hands-on dbt and Google Cloud Platform experience.

Responsibilities

• Build ingestion across multiple providers and formats, handling concurrency, rate limits, retries, and failures.
• Orchestrate scheduled jobs and historical backfills with a workflow engine; implement event-driven processing with queues, dead-letter handling, and controlled replay.
• Preserve original source payloads in cloud storage and transform them into raw, standardized, and curated data layers.
• Design idempotent updates and safe recovery paths to support reprocessing without duplicates or corruption.
• Create data contracts, automated validation, pipeline tests, and monitoring for freshness and failures.
• Design and optimize PostgreSQL schemas, queries, and incremental loading for reliable downstream delivery.
• Integrate provider enrichment and structured extraction from text using language models.

Skills

SQL proficiency

Python programming

Asynchronous processing

Data modeling

Tools

Airflow

Dagster

Prefect

Temporal

dbt

Google Cloud Platform

PostgreSQL

Job description

silver seeks a data platform engineer to own the architecture, implementation, and production operation of a data platform consolidating data from APIs, webhooks, files, and web scraping. You will evolve serial workflows into concurrent, event-driven intake and orchestrated batch processing, delivering traceable records to downstream products.

You will build ingestion across providers, orchestrate jobs with a workflow engine, preserve source payloads in cloud storage, and implement robust,