Data Pipeline Engineer — Apache Airflow (Astronomer) & Python

Triade LLC; United States; A distancia; USD 83.000 - 165.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Fully remote

Descripción de la vacante

Triade LLC is seeking data pipeline engineers for a large migration program in a highly controlled banking environment. This is a fully remote engagement for a US-based client, with work hours aligned to United States business hours and compensation paid in USD.

You will design and operate ETL/ELT pipelines in Python and PySpark, author and run Apache Airflow DAGs, deploy to production via CI/CD, migrate existing workloads, own observability, and communicate clearly in written English to the

Formación

• Expert-level Apache Airflow in production.
• Strong Python and PySpark for data engineering.
• You have deployed to production yourself, through CI/CD.
• SQL and warehouse/lakehouse patterns.
• Comfort working under strict security controls.

Responsabilidades

• Create ETL/ELT pipelines in Python and PySpark — ingestion, transformation, orchestration, retries, idempotency, backfills.
• Author and operate Apache Airflow DAGs — dependencies, sensors, dynamic task mapping, SLAs, alerting.
• Deploy to production — CI/CD, code review, environment promotion, release discipline in a controlled, audited environment.
• Migrate and implement at volume — stand up a large backlog of jobs with repeatability.
• Own operational quality — monitoring, logging, cost and runtime, incident response.
• Communicate in writing, in English — pull requests, runbooks, status to the client.

Conocimientos

Airflow production

Python

PySpark

SQL

CI/CD

Herramientas

Astronomer

Starburst

Trino

Iceberg

dbt

Kafka

AWS

GCP

Descripción del empleo

About the role

Triade is a data and analytics consultancy working across the United States, Brazil and Mexico. We are adding two data pipeline engineers to a large pipeline build-and-migrate programme for a financial-services client, delivered through our platform partner.

A controlled banking environment — please read before applying

All work is done inside the client's highly controlled and monitored banking environment.

• No AI-assisted development. AI coding assistants and generative AI tools (GitHub Copilot, ChatGPT or similar) may not be used for any work on this engagement. You write the code yourself.
• No copy and paste into or out of the environment.
• No file upload or download — nothing leaves or enters the environment except through the client's approved channels.
• Activity is monitored. Sessions and actions in the environment are logged and reviewed.

A breach of these rules can end the engagement immediately, damage your professional reputation and Triade's, and expose the people involved to legal action. If you cannot work this way, this role is not for you.

What you will do

• Create ETL/ELT pipelines in Python and PySpark — ingestion, transformation, orchestration, retries, idempotency, backfills.
• Author and operate Apache Airflow DAGs — dependencies, sensors, dynamic task mapping, SLAs, alerting. Keep a daily chain running when one task fails.
• Deploy to production — CI/CD, code review, environment promotion, release discipline in a controlled, audited environment.
• Migrate and implement at volume — a large backlog of jobs to stand up; repeatability and patterns matter more than bespoke cleverness.
• Own operational quality — monitoring, logging, cost and runtime, incident response on your own pipelines.
• Communicate in writing, in English — pull requests, runbooks, status to the client.

What you need

• Expert-level Apache Airflow in production. Not coursework, not "exposure" — you have owned DAGs that other people depended on.
• Strong Python and PySpark for data engineering — clean, tested, reviewed code, written without AI assistance.
• You have deployed to production yourself, through CI/CD, in an environment with change control.
• SQL and comfort with warehouse/lakehouse patterns.
• Comfort working under strict security controls — no AI tools, no copy and paste, no file transfer, monitored sessions.
• English at a working professional level. You will write for and speak to a financial services client. This one is not negotiable.
• Available to start in early November and to work United States business hours. There is no extended ramp on this engagement.

Nice to have

• Astronomer / Astro specifically — managed Airflow, Astro CLI, deployment workflows.
• Astronomer or Apache Airflow certification.
• Starburst, Trino, Iceberg, or similar federated query and lakehouse technology.
• Financial services or another regulated environment — ideally inside a bank's controlled environment.
• dbt, Kafka; AWS, Azure or GCP.
• Experience on a large migration rather than greenfield only.

How we work

• Fully remote, United States business hours.
• Written status over verbal — a decision that is not written down did not happen.
• You work alongside data architects and platform engineers, and with partner delivery teams.

Details

Independent contractor through Triade LLC, paid in US dollars, 6 to 12 months, hourly, extendable.

Tell us your expected hourly rate and currency when you apply.

Triade is an equal opportunity company. We consider all qualified applicants without regard to any characteristic protected by applicable law.