Remote Lead Data Engineer: OLAP & AWS Data Lake

AgileEngine; Sur; Presencial; COP 374.895.000 - 562.342.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Competitive compensation

Flexibility

Meaningful, modern projects

Collaborative culture

Well-being & support

Descripción de la vacante

AgileEngine is seeking a Lead Data Engineer to own the data pipeline and analytics architecture for a high-volume marketing analytics platform. You will shape partitioning strategy, file formats, and near-real-time processing on an S3-backed data lake.

Design ETL pipelines, define Airflow DAGs, and drive the AWS data stack with Athena and EKS, while guiding senior developers and upholding engineering standards.

Formación

• Minimum 7+ years of engineering experience designing ETL pipelines and data architectures.
• Hands-on OLAP data stack experience with tools like Athena, Trino/Presto, or Snowflake.
• Experience designing object-storage data lakes on S3 with Parquet/ORC formats.
• Strong Python development skills and REST/GraphQL API exposure.

Responsabilidades

• Design and own ETL pipelines extracting data from internal databases and APIs at scale.
• Define partitioning, file formats, and near-real-time processing for OLAP workloads.
• Own Airflow DAG design, dependencies, and triggering strategies with DevOps partners.
• Lead senior developers, enforce code quality and architectural standards.
• Collaborate across teams to translate business needs into scalable data solutions.

Conocimientos

ETL design

Python

Airflow

AWS

Big data architecture

REST/GraphQL

Leadership

Ambiguity handling

English communication

Educación

Bachelor's degree in CS or related field

Herramientas

Airflow

Spark SQL

Hadoop/Spark

Airflow & Dagster

PostgreSQL

Docker

Kubernetes

Athena/Presto/Trino

S3/Parquet

Descripción del empleo

AgileEngine is seeking a Lead Data Engineer to own the data pipeline and analytics architecture for a high-volume marketing analytics platform. You will shape partitioning strategy, file formats, and near-real-time processing on an S3-backed data lake.

Design ETL pipelines, define Airflow DAGs, and drive the AWS data stack with Athena and EKS, while guiding senior developers and upholding engineering standards.