Lead Data Engineer - Remote, Flexible Hours, Growth Budget

AgileEngine, LLC.; Rosarito; Presencial; MXN 1.100.000 - 1.800.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth opportunities

Competitive compensation

Remote work

Meaningful projects

Collaborative culture

Well-being programs

Descripción de la vacante

AgileEngine, LLC. is seeking a Lead Data Engineer to own the data pipeline and analytical architecture for a large-volume analytics platform.

You will shape partitioning strategies, file formats, and near-real-time processing for OLAP workloads on an S3-backed data lake, while designing ETL pipelines and governing Airflow-based DAGs. You will lead a team of senior developers, enforce code quality, and partner with DevOps to align on functional and non-functional requirements.

Formación

• 7+ years of engineering experience designing and implementing ETL pipelines for large-volume data systems.
• Hands-on experience with OLAP-style architectures and the ability to work with Athena, Trino/Presto, BigQuery, Snowflake, Spark SQL or equivalent.
• Experience designing object-storage-backed data lakes (S3 or similar) with partitioning, Parquet/ORC formats, and cost/performance tradeoffs.
• Deep familiarity with DAG-style workflow orchestration (Airflow or comparable like Dagster, Prefect, Luigi, Step Functions).
• Proficiency across AWS data stack: S3, serverless engines (Athena), and Kubernetes (EKS).
• Backend proficiency in Python (FastAPI or Flask).
• Comfort with REST and GraphQL APIs.
• Docker and PostgreSQL in transactional/app layers.
• Mac/Linux terminal proficiency.
• Hands-on use of AI-assisted development tools with sound judgment to maintain long-term maintainability.
• Strong soft skills to defend technical opinions and drive scalable solutions.
• Comfort with ambiguity; ability to define problems and frame work before solving.

Responsabilidades

• Design and own ETL pipelines that extract, transform, and validate data at scale.
• Make architectural calls on partitioning, file formats, and near-real-time processing for OLAP workloads.
• Own scheduling and structure of DAGs in Airflow; drive architectural discussions (deployment not in scope).
• Drive AWS data stack usage and collaborate with DevOps to clarify requirements.
• Review PRs and enforce code quality standards.
• Guide senior developers and align with established engineering practices.

Conocimientos

ETL pipeline design

OLAP data architecture

Python

Airflow

AWS data stack

S3/Data Lake

Parquet/ORC

Docker

PostgreSQL

REST/GraphQL

Linux/Mac terminal

AI-assisted development tools

Leadership

Ambiguity handling

Herramientas

Dagster

Prefect

Kubernetes

Trino/Presto

Athena

Spark SQL

Snowflake

FastAPI

Descripción del empleo

AgileEngine, LLC. is seeking a Lead Data Engineer to own the data pipeline and analytical architecture for a large-volume analytics platform.

You will shape partitioning strategies, file formats, and near-real-time processing for OLAP workloads on an S3-backed data lake, while designing ETL pipelines and governing Airflow-based DAGs. You will lead a team of senior developers, enforce code quality, and partner with DevOps to align on functional and non-functional requirements.