Tech Lead Data Engineer — OLAP, Airflow & AWS Leadership

AgileEngine; Mexico; Hybrid; PHP 9,124,000 - 12,774,000; Full time

Benefits offered by this job

Professional growth

Competitive USD-based compensation

Exciting projects

Flextime

Job summary

AgileEngine seeks a Tech Lead Data Engineer to own data pipelines and the analytical architecture for a large-volume marketing analytics platform. You will define partitioning strategies, file formats, and schema, while steering near-real-time processing for OLAP workloads on an S3-backed data lake.

You will design and govern ETL pipelines, define DAG-based orchestration with Airflow, drive AWS data stack including Athena and EKS, and lead a team of senior developers while upholding code quality

Qualifications

• 7+ years of engineering experience designing and implementing ETL pipelines for large-volume data systems.
• Hands-on OLAP-style data architecture with serverless query engines; Athena, Trino/Presto, BigQuery, Snowflake, Spark SQL, or similar.

Responsibilities

• Design and own ETL pipelines that extract, transform, and validate data at scale.
• Make architectural decisions on partitioning, file formats, schema, and near-real-time processing for OLAP workloads.
• Own the design of scheduled batch workflows (DAGs) on the client’s Airflow setup, defining pipeline structure and triggering strategy.
• Drive use of the client's AWS data stack (S3, Athena, EKS) and collaborate with DevOps on infra requirements.
• Review PRs and enforce code quality.
• Guide senior developers and align with client engineering practices.

Skills

7+ years of engineering experience

OLAP-style architecture

Athena

Trino/Presto

BigQuery

Snowflake

Spark SQL

ClickHouse

Parquet/ORC

Python

FastAPI/Flask

REST

GraphQL

Docker

PostgreSQL

Mac/Linux terminal

Airflow

Dagster/Prefect/Luigi/Step Functions

AWS data stack (S3, Athena, EKS)

Airflow architecture leadership

AI-assisted development tools (Claude)

Tools

Airflow

Dagster

Prefect

Luigi

Step Functions

Job description

AgileEngine seeks a Tech Lead Data Engineer to own data pipelines and the analytical architecture for a large-volume marketing analytics platform. You will define partitioning strategies, file formats, and schema, while steering near-real-time processing for OLAP workloads on an S3-backed data lake.

You will design and govern ETL pipelines, define DAG-based orchestration with Airflow, drive AWS data stack including Athena and EKS, and lead a team of senior developers while upholding code quality