Lead Data Engineer: OLAP Analytics & Data Lake

AgileEngine, LLC.; Capital; Presencial; COP 120.000.000 - 210.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Competitive compensation

Flexibility — 100% remote with flexi‑t

Meaningful, modern projects with top-?

Descripción de la vacante

AgileEngine LLC is seeking a Lead Data Engineer to own the data pipeline and analytical architecture for a high-volume marketing analytics platform. You will shape partitioning, file formats, and near-real-time processing on an S3-backed data lake, while guiding ETL pipelines and Airflow DAGs.

You will drive the AWS data stack, collaborate with DevOps, review PRs, and lead senior developers to uphold engineering standards.

Formación

• 7+ years of engineering experience designing and implementing ETL pipelines for large-volume data systems.
• Hands-on OLAP-style analytics architecture experience with Athena/Presto/BigQuery/Snowflake/Spark SQL.
• Experience with object-storage-backed data lakes on S3 or equivalent, including Parquet/ORC and partitioning strategies.
• Deep familiarity with DAG-style workflow orchestration (Airflow or similar).
• Strong AWS data stack experience (S3, Athena, EKS/Kubernetes).
• Backend proficiency in Python (FastAPI or Flask) and REST/GraphQL.
• Docker and PostgreSQL for transactional layer.
• Mac/Linux terminal proficiency.
• Experience with AI-assisted development tools and responsible AI usage.
• Strong soft skills and ability to defend technical positions.
• Comfort with ambiguity and underspecified problems.

Responsabilidades

• Design and own ETL pipelines that extract, transform, and validate data at scale.
• Make architectural calls on partitioning, file formats, and near-real-time processing for large-volume OLAP data systems.
• Own the design of scheduled batch workflows (DAGs) on the Airflow setup, guiding pipeline structure and dependencies.
• Drive use of the AWS data stack (S3-backed data lake, Athena, EKS/Kubernetes) in collaboration with DevOps.
• Review PRs and enforce code quality standards.
• Guide senior developers and ensure alignment with established engineering practices.

Conocimientos

7+ years engineering experience

OLAP data architecture

Object-storage data lakes

Airflow and DAGs

AWS data stack

Python backend

REST and GraphQL

Docker and PostgreSQL

Mac/Linux terminal

AI-assisted development tools

Ambiguity tolerance

Herramientas

Airflow

Docker

PostgreSQL

Kubernetes

Descripción del empleo

AgileEngine LLC is seeking a Lead Data Engineer to own the data pipeline and analytical architecture for a high-volume marketing analytics platform. You will shape partitioning, file formats, and near-real-time processing on an S3-backed data lake, while guiding ETL pipelines and Airflow DAGs.

You will drive the AWS data stack, collaborate with DevOps, review PRs, and lead senior developers to uphold engineering standards.