Senior Data Engineer, LATAM

Clera; United States; Remoto; USD 80.000 - 120.000; Tempo pieno

Descrizione del lavoro

Clera Labs, Inc. is seeking a senior Data Platform Engineer to own a multi-source data platform from ingestion to production. You will design, implement, and operate end-to-end pipelines, focusing on reliability, correctness, and scalability across APIs, webhooks, files, and web scraping.

Emphasis on dbt, PostgreSQL, and cloud-native orchestration with Airflow, Dagster, or Prefect. You will work remotely with a distributed team and leverage modern tooling to maintain data quality, lineage, and

Competenze

• Own data platform end-to-end from ingestion to production.
• Experience with multi-source ingestion and orchestration.
• Strong Python with async processing.
• Strong SQL and PostgreSQL data modeling.
• Experience with dbt and GCP.
• Experience with event-driven systems and messaging.

Mansioni

• Design a multi-source data platform for concurrent ingestion, event-driven processing, and orchestrated batch workflows.
• Build intake pipelines for APIs, webhooks, files, and web scraping, with rate limits and robust failure handling.
• Orchestrate scheduled workflows and historical backfills using tools such as Airflow, Dagster, or Prefect.
• Develop messaging and task queue systems with retries, dead-letter queues, and controlled replay.
• Preserve source payloads in Google Cloud Storage and transform data with dbt into raw, standardized, and curated layers.
• Implement idempotent upserts, data contracts, automated validation, pipeline tests, and monitoring for freshness and failures.

Conoscenze

Python

Async I/O

SQL

PostgreSQL

Data modeling

Pipelines

Workflow orchestration

Airflow

Dagster

Prefect

Temporal

LLM-based extraction

Strumenti

dbt

Google Cloud Platform

Airflow

Dagster

Prefect

Temporal

aiohttp

HTTPX

Descrizione del lavoro

About the Role

Join a data infrastructure and analytics team building a platform that turns information from multiple sources into reliable, current company intelligence. You will own the platform end-to-end, from architecture and ingestion through production operations.

What You'll Do

• Design a multi-source data platform for concurrent ingestion, event-driven processing, and orchestrated batch workflows.

• Build intake pipelines for APIs, webhooks, files, and web scraping, with rate limits and robust failure handling.

• Orchestrate scheduled workflows and historical backfills using tools such as Airflow, Dagster, or Prefect.

• Develop messaging and task queue systems with retries, dead-letter queues, and controlled replay.

• Preserve source payloads in Google Cloud Storage and transform data with dbt into raw, standardized, and curated layers.

• Implement idempotent upserts, data contracts, automated validation, pipeline tests, and monitoring for freshness and failures.

• Design and optimize PostgreSQL schemas and loading patterns, and integrate enrichment and LLM-based structured extraction.

What We're Looking For

• At least 5 years building and operating production data platforms across multiple sources, with ownership from architecture through deployment and ongoing operations.

• Hands-on production experience with dbt and Google Cloud Platform, and workflow orchestration using Airflow, Dagster, Prefect, Temporal, or a comparable tool. Airflow is preferred.

• Strong Python skills, including asynchronous HTTP and concurrent processing with asyncio, aiohttp, or HTTPX.

• Strong SQL and PostgreSQL skills, including data modeling, query optimization, and reliable incremental loading.

• Experience building event-driven systems with messaging or task queues, asynchronous processing, and failure recovery.

• Experience with data contracts, raw and curated storage, retention policies, automated testing, and production monitoring.

• Helpful experience includes web scraping, entity resolution, company and people data enrichment, LLM-based extraction, and building platforms from scratch in small teams.

Compensation & Benefits

Base salary: $80,000 to $120,000 USD annually. Visa sponsorship is not available.

Location

Fully remote for candidates based in Latin America.