Data Pipeline Engineer — Airflow & PySpark (Remote)

Triade LLC; United States; Teletrabalho; USD 83 000 - 165 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Fully remote

Resumo da oferta

Triade LLC is seeking data pipeline engineers for a large migration program in a highly controlled banking environment. This is a fully remote engagement for a US-based client, with work hours aligned to United States business hours and compensation paid in USD.

You will design and operate ETL/ELT pipelines in Python and PySpark, author and run Apache Airflow DAGs, deploy to production via CI/CD, migrate existing workloads, own observability, and communicate clearly in written English to the

Qualificações

• Expert-level Apache Airflow in production.
• Strong Python and PySpark for data engineering.
• You have deployed to production yourself, through CI/CD.
• SQL and warehouse/lakehouse patterns.
• Comfort working under strict security controls.

Responsabilidades

• Create ETL/ELT pipelines in Python and PySpark — ingestion, transformation, orchestration, retries, idempotency, backfills.
• Author and operate Apache Airflow DAGs — dependencies, sensors, dynamic task mapping, SLAs, alerting.
• Deploy to production — CI/CD, code review, environment promotion, release discipline in a controlled, audited environment.
• Migrate and implement at volume — stand up a large backlog of jobs with repeatability.
• Own operational quality — monitoring, logging, cost and runtime, incident response.
• Communicate in writing, in English — pull requests, runbooks, status to the client.

Conhecimentos

Airflow production

Python

PySpark

SQL

CI/CD

Ferramentas

Astronomer

Starburst

Trino

Iceberg

dbt

Kafka

AWS

GCP

Descrição da oferta de emprego

Triade LLC is seeking data pipeline engineers for a large migration program in a highly controlled banking environment. This is a fully remote engagement for a US-based client, with work hours aligned to United States business hours and compensation paid in USD.

You will design and operate ETL/ELT pipelines in Python and PySpark, author and run Apache Airflow DAGs, deploy to production via CI/CD, migrate existing workloads, own observability, and communicate clearly in written English to the