Lakehouse Data Engineer — Spark, Delta & Streaming

AgileEngine; Santiago de Querétaro; A distancia; MXN 600.000 - 900.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Competitive compensation

100% remote with flexible hours

Modern projects with global teams

Collaborative culture

Well-being & support

Descripción de la vacante

AgileEngine is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You’ll design and optimize batch and streaming pipelines using PySpark, Delta Lake, and Databricks Workflows, following a medallion architecture for analytics and ML.

You will also migrate legacy workloads, ensure data parity, and leverage AI tools to accelerate development, testing, and documentation while maintaining high coding standards.

Formación

• 3+ years in data engineering with Spark and cloud data architectures.
• Hands-on Databricks, PySpark, and Delta Lake experience.
• Advanced SQL and Python with dimensional and Lakehouse data modeling.
• Experience with streaming using Structured Streaming and Kafka/Event Hubs.
• Experience with orchestration tools (Airflow, Databricks Workflows, or Azure Data Factory).
• Familiarity with Unity Catalog and data governance controls.
• Proficient in English (upper-intermediate) for collaboration.

Responsabilidades

• Design, build, and operate batch and streaming pipelines on Databricks with PySpark and Delta Lake.
• Model and maintain a bronze/silver/gold medallion architecture for analytics and ML.
• Migrate legacy ETL workloads to Lakehouse with minimal disruption.
• Leverage Claude or Copilot to accelerate development and testing.
• Write clean Python and SQL; ensure code quality via reviews and docs.
• Optimize Spark jobs and Delta tables for performance and cost.
• Implement data quality, lineage, and governance checks with Unity Catalog.

Conocimientos

Spark

Databricks

PySpark

Delta Lake

SQL

Python

Structured Streaming

Workflow Orchestration

Data Modeling

Unity Catalog

dbt

Descripción del empleo

AgileEngine is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You’ll design and optimize batch and streaming pipelines using PySpark, Delta Lake, and Databricks Workflows, following a medallion architecture for analytics and ML.

You will also migrate legacy workloads, ensure data parity, and leverage AI tools to accelerate development, testing, and documentation while maintaining high coding standards.