Lead Data Engineer — Remote, Growth & Modern Analytics

AgileEngine; Almoloya de Juárez; Presencial; MXN 900.000 - 1.350.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Competitive compensation

Remote work with flexible hours

Meaningful, modern projects

Collaborative culture

Well-being & support

Descripción de la vacante

AgileEngine seeks a Lead Data Engineer to own the data pipeline and analytics layer for a large-volume marketing analytics platform. You will shape partitioning, file formats, and near-real-time processing for OLAP workloads on an S3 data lake.

You will design ETL pipelines, govern schemas, and lead a team of senior developers while upholding code quality and engineering best practices.

Formación

• 7+ years of engineering experience designing ETL pipelines for large data systems.
• Hands-on OLAP data architecture with Athena/Presto/Snowflake or similar.
• Experience with object-storage data lakes (S3) and Parquet/ORC formats.
• Deep knowledge of DAG-based workflow orchestration (Airflow or equivalent).
• AWS data stack expertise (S3, Athena, EKS) and DevOps collaboration.
• Backend in Python and REST/GraphQL familiarity.

Responsabilidades

• Design and own ETL pipelines for data extraction from internal databases and external APIs.
• Make architectural calls on partitioning, formats, and real-time processing for OLAP data lakes.
• Own DAG design, scheduling, and triggering of batch workflows via Airflow.
• Drive AWS data stack usage and collaborate with DevOps on infra needs.
• Review PRs and enforce code quality standards.
• Lead senior developers and align with established engineering practices.

Conocimientos

ETL pipelines

OLAP architecture

Python

Airflow

AWS data stack

S3 data lake

Data governance

Airflow orchestration

Dagster/Prefect/Luigi

REST/GraphQL

Docker

PostgreSQL

Python in ML/AI tooling

Herramientas

Airflow

Dagster

Prefect

Luigi

Step Functions

Python

Docker

PostgreSQL

Athena

EKS/Kubernetes

Descripción del empleo

AgileEngine seeks a Lead Data Engineer to own the data pipeline and analytics layer for a large-volume marketing analytics platform. You will shape partitioning, file formats, and near-real-time processing for OLAP workloads on an S3 data lake.

You will design ETL pipelines, govern schemas, and lead a team of senior developers while upholding code quality and engineering best practices.