Lead Data Engineer — AWS Data Lake & OLAP Architect

AgileEngine, LLC.; Medellín; Presencial; COP 140.000.000 - 220.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Competitive compensation

Flexibility – 100% remote

Meaningful, modern projects

Collaborative culture

Well-being & support

Descripción de la vacante

AgileEngine is seeking a Lead Data Engineer to own data pipelines and the analytical layer for a large-volume marketing analytics platform. You will shape partitioning strategies, file formats, and near-real-time processing on an S3-backed data lake, while governing ETL pipelines and DAG orchestration with Airflow.

You will drive the AWS data stack (Athena, EKS) and collaborate with a DevOps partner to meet functional and non-functional requirements, mentoring senior developers and ensuring high

Formación

• 7+ years of engineering experience with ETL pipelines and large-volume data systems.
• Experience with OLAP-style data architectures and lakehouse concepts.
• Backend programming in Python (FastAPI or Flask) and REST/GraphQL APIs.
• Hands-on with object-storage data lakes (S3) and partitioning/file formats such as Parquet/ORC.

Responsabilidades

• Design and own ETL pipelines at scale for OLAP workloads.
• Make architectural calls on partitioning, file formats, and near-real-time processing.
• Design scheduled batch workflows on Airflow and define dependencies.
• Drive AWS data stack (S3, Athena, EKS) alongside DevOps.
• Review PRs and enforce code quality across the team.
• Lead senior developers and uphold engineering standards.

Conocimientos

ETL pipelines

OLAP architecture

Python

REST/GraphQL

Airflow

AWS data stack

Docker

PostgreSQL

Linux/macOS

Herramientas

Airflow (or similar)

AWS

PostgreSQL

Descripción del empleo

AgileEngine is seeking a Lead Data Engineer to own data pipelines and the analytical layer for a large-volume marketing analytics platform. You will shape partitioning strategies, file formats, and near-real-time processing on an S3-backed data lake, while governing ETL pipelines and DAG orchestration with Airflow.

You will drive the AWS data stack (Athena, EKS) and collaborate with a DevOps partner to meet functional and non-functional requirements, mentoring senior developers and ensuring high