Lead Data Engineer - Remote OLAP/Data Lake Architect

AgileEngine; Rosarito; Presencial; MXN 2.212.000 - 3.063.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth opportunities

Remote work

Annual learning budget

Competitive compensation

Well-being programs

Descripción de la vacante

AgileEngine is seeking a Lead Data Engineer to own the data pipeline and analytical architecture for a high-volume marketing analytics platform. You’ll define partitioning, file formats, and near-real-time processing for OLAP workloads on an S3-backed data lake, and lead a team of senior developers.

You will design ETL pipelines, orchestrate with Airflow, and drive the AWS data stack including Athena and EKS, while enforcing code quality and engineering practices.

Formación

• 7+ years of engineering experience with ETL pipelines and large-volume data systems.
• Hands-on OLAP-focused data architecture experience with applicable tools.
• Experience designing data lakes on object storage with partitioning and Parquet/ORC formats.
• Deep familiarity with DAG-based workflow orchestration (Airflow or comparable).

Responsabilidades

• Design and own ETL pipelines at scale across internal databases and external APIs.
• Make architectural calls on partitioning, file formats, and near-real-time processing for OLAP workloads.
• Design and govern Airflow DAGs and pipeline structure; collaborate with DevOps on requirements.
• Drive AWS data stack (S3, Athena, EKS) and partner with DevOps for infra decisions.
• Review PRs and enforce code quality standards.
• Guide senior developers and align with established engineering practices.

Conocimientos

ETL design

OLAP architecture

Airflow

AWS data stack

Python

REST/GraphQL

Docker

PostgreSQL

Kubernetes

Data lake (S3)

AI-assisted development

Ambiguity handling

Herramientas

Airflow

Athena

Trino/Presto

BigQuery

Snowflake

Spark SQL

Parquet/ORC

S3

Descripción del empleo

AgileEngine is seeking a Lead Data Engineer to own the data pipeline and analytical architecture for a high-volume marketing analytics platform. You’ll define partitioning, file formats, and near-real-time processing for OLAP workloads on an S3-backed data lake, and lead a team of senior developers.

You will design ETL pipelines, orchestrate with Airflow, and drive the AWS data stack including Athena and EKS, while enforcing code quality and engineering practices.