Fundable - Senior Data Engineer

OpenKrill; United States; Remote; USD 140,000 - 210,000; Full time

Job summary

Fundable is seeking a senior data engineer to own the data platform from architecture to production. You will build a concurrent ingestion layer for providers, webhooks, files, and scraping, and orchestrate both batch and event-driven workflows using Airflow, Dagster, or Prefect.

You will preserve raw payloads in Google Cloud Storage and transform data with dbt across medallion layers. You will implement idempotent upserts, facilitate safe retries, and design robust data contracts.

Qualifications

• Experience building multi-source production data platforms from architecture to operation.
• Production workflow orchestration with Airflow, Dagster, Prefect, or Temporal; Airflow preferred.
• Hands-on dbt and Google Cloud Platform experience.
• Event-driven systems with messaging or task queues and reliable retries.
• Strong SQL/PostgreSQL data modeling, loading patterns, and data contracts.
• Strong Python with asynchronous processing and concurrency.

Responsibilities

• Own Fundable’s data platform architecture and production readiness.
• Ingest data from provider APIs, webhooks, files, and scraping with concurrency.
• Orchestrate batch and event-driven workflows using Airflow, Dagster, or similar.
• Preserve raw payloads in GCS and transform with dbt across medallion layers.
• Ensure data quality, validation, and monitoring of freshness and failures.

Skills

SQL & PostgreSQL

Python

Data modeling

ETL pipelines

Data quality

Tools

Airflow

dbt

Dagster

Prefect

BigQuery

GCS

Job description

About Fundable

Fundable is a startup data platform that helps investors and other teams discover and research companies. We bring together information on companies, investors, funding activity, and people to make that research easier.

Role overview

We’re looking for a senior data engineer to own Fundable’s data platform from architecture through production, reliably turning high volumes of data from provider APIs, webhooks, files, and web scraping into trusted records for our products.

Today, our enrichment and extraction workflows largely run serially inside a container. You’ll lead the move to a platform that supports concurrent processing, event-driven intake, and orchestrated batch workflows.

What you’ll own

• Multi-source intake: Build concurrent ingestion for provider APIs, webhooks, files, and scraping, with rate limits and failure handling.
• Batch and event-driven execution: Orchestrate scheduled workflows and historical backfills using tools such as Airflow, Dagster, or Prefect. Build event-driven intake with Pub/Sub or Cloud Tasks, including retries, dead-letter queues, and controlled replay of failed work.
• Raw-to-curated data: Preserve original payloads, including provider JSON, in Google Cloud Storage (GCS) before transforming them with dbt. Establish medallion layers for raw, standardized, and curated data.
• Correctness through recovery: Implement idempotent upserts and safe retries, replay, and historical backfills that avoid duplicate records or data corruption. Make data traceable to its source and safe to reprocess.
• Data quality and delivery: Define data contracts, build automated validation and pipeline tests, and monitor freshness and failures. Design and optimize PostgreSQL schemas, queries, and loading patterns so downstream products receive dependable data.
• Extraction and enrichment: Integrate provider enrichment and LLM-based structured extraction from text into the platform.

Must-have experience

• Experience building multi-source production data platforms from zero to one, from initial architecture and pipeline implementation through deployment and ongoing operation.
• Production experience with workflow orchestration tools such as Airflow, Dagster, Prefect, or Temporal, including dependencies, retries, and failure recovery. Airflow experience is preferred.
• Hands-on production experience with dbt and GCP.
• Experience building event-driven systems with messaging or task queues, including asynchronous processing and failure recovery.
• Strong SQL and PostgreSQL skills, including data modeling, query optimization, and reliable incremental loading. Experience with data contracts, raw and curated storage, retention policies, and automated quality checks.
• Strong Python skills, including asynchronous HTTP and concurrent processing with tools such as asyncio, aiohttp, or HTTPX.
• Hands-on web scraping experience with tools such as Scrapy, Playwright, Selenium, or Beautiful Soup, including concurrent scraping, rate limits, and failure handling.

Nice-to-have experience

• Experience working in startups or small engineering teams, with broad ownership and changing priorities.
• Entity resolution experience is especially valuable: matching company and people records across sources and reconciling duplicates into consistent entities.
• LLM-based structured extraction or company and people data enrichment.
• Familiarity with BigQuery and Dataform.

Interview Process

• Silver Screening Interview
• Offline Case Study
• Client Behavioral Interview
• Client Case Study Interview
• Client Final Round Interview