Lead Data Engineer — Remote, Growth & Impact

AgileEngine; Capital; Presencial; COP 170.000.000 - 230.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Competitive compensation

100% remote with flexible hours

Meaningful, modern projects

Collaborative culture

Well-being & support

Descripción de la vacante

AgileEngine, a leading software and data engineering firm, seeks a Lead Data Engineer to own data pipelines and analytics architecture for a high-volume marketing analytics platform. You will shape partitioning strategies, file formats, schema design, and near-real-time processing for OLAP workloads built on an S3-backed data lake.

You will guide ETL design, DAG orchestration with Airflow, and AWS data services, leading senior developers while upholding coding standards.

Formación

• 7+ years of engineering experience with ETL pipelines and large-volume data systems.
• Hands-on OLAP-style data architecture experience with Athena, Trino/Presto, BigQuery, Snowflake, Spark SQL, ClickHouse or similar.
• Experience designing data lakes on object storage (S3 or equivalent) with partitioning, Parquet/ORC formats, cost-performance considerations.
• Deep familiarity with DAG-style workflow definition and scheduling (Airflow or comparable like Dagster/Prefect/Luigi/Step Functions).
• Hands-on AWS data ecosystem experience (S3 data lakes, serverless query engines like Athena, EKS/Kubernetes).
• Strong backend in Python, including FastAPI or Flask; REST and GraphQL.
• Experience with Docker and PostgreSQL in production.
• Comfortable in Mac/Linux terminals; able to guide AI-assisted development tools responsibly.
• Excellent communication and leadership to drive scalable, maintainable solutions.

Responsabilidades

• Design and own ETL pipelines for large-volume data systems.
• Make architectural decisions on partitioning, file formats, and near-real-time processing for OLAP workloads.
• Own DAG-based batch workflows on the client’s Airflow setup and drive architecture discussions.
• Drive use of the AWS data stack (S3, Athena, EKS) and collaborate with DevOps on requirements.
• Review PRs and enforce code quality; mentor senior developers and align with client engineering practices.

Conocimientos

Leadership

English (upper-intermediate)

Python backend

REST/GraphQL

AI-assisted development tools

Data architecture

Soft skills

Herramientas

Airflow

Athena

Trino/Presto

BigQuery

Snowflake

Spark SQL

Parquet/ORC

S3

EKS/Kubernetes

PostgreSQL

Docker

Descripción del empleo

AgileEngine, a leading software and data engineering firm, seeks a Lead Data Engineer to own data pipelines and analytics architecture for a high-volume marketing analytics platform. You will shape partitioning strategies, file formats, schema design, and near-real-time processing for OLAP workloads built on an S3-backed data lake.

You will guide ETL design, DAG orchestration with Airflow, and AWS data services, leading senior developers while upholding coding standards.