Senior Data Engineer Lead - Remote & AWS Data Lake

INGEPSY; Sucre; Presencial; COP 120.000.000 - 200.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Competitive compensation

Flexibility: 100% remote

Meaningful projects

Collaborative culture

Well-being & support

Descripción de la vacante

AgileEngine is seeking a Lead Data Engineer to own the data pipeline and analytical architecture for a large-scale marketing analytics platform. You will design partitioning, file formats, schema, and near-real-time processing on an S3-backed data lake while guiding senior developers and enforcing code quality.

You will govern ETL pipelines, orchestrate DAGs with Airflow, and collaborate with DevOps across AWS data services (Athena, EKS).

Formación

• 7+ years of engineering experience with ETL pipelines and large-volume data systems.
• Hands-on OLAP-style analytics with Athena/Trino/BigQuery/Snowflake/Spark SQL.
• Design against a data lake on object storage (S3) with Parquet/ORC formats.
• Deep knowledge of DAG-based workflows and Airflow (or alternatives).
• Experience AWS data stack: S3, Athena, EKS/Kubernetes; work with DevOps.
• Strong Python backend skills (FastAPI/Flask); REST/GraphQL.
• Docker and PostgreSQL for transactional/app layers.
• Hands-on use of AI-assisted tools with responsible use guidance.
• Strong communication and leadership; defend technical stance.
• Upper-intermediate English.

Responsabilidades

• Design and own ETL pipelines at scale.
• Architect partitioning, file formats, and near-real-time processing for OLAP workloads.
• Own DAG-based workflow design on Airflow, scaffolding pipelines and scheduling.
• Drive AWS data stack usage with DevOps for scalable infrastructure.
• Review PRs and enforce code quality.
• Lead senior developers and align with client engineering practices.

Conocimientos

ETL pipelines design

OLAP data architecture

Airflow DAG orchestration

AWS data stack (S3, Athena, EKS)

Python backend (FastAPI/Flask)

REST & GraphQL

Docker & PostgreSQL

Unix/Linux CLI

AI-assisted development tools

Leadership & technical opinion

English (upper-intermediate)

Herramientas

Airflow

Trino/Presto/Spark SQL

Athena

Kubernetes / EKS

Docker

PostgreSQL

Descripción del empleo

AgileEngine is seeking a Lead Data Engineer to own the data pipeline and analytical architecture for a large-scale marketing analytics platform. You will design partitioning, file formats, schema, and near-real-time processing on an S3-backed data lake while guiding senior developers and enforcing code quality.

You will govern ETL pipelines, orchestrate DAGs with Airflow, and collaborate with DevOps across AWS data services (Athena, EKS).