Remote Data Engineer — Lakehouse, PySpark & AI-Accelerated Pipelines

AgileEngine, LLC.; Región Centro; Presencial; MXN 420.000 - 820.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Competitive compensation

Flexibility: 100% remote with flexible

Meaningful modern projects

Collaborative culture

Well-being & support

Descripción de la vacante

AgileEngine, LLC. is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse.

You will build batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers. Join a team leveraging AI tools like Claude and GitHub Copilot to accelerate development while ensuring data quality and governance.

Formación

• 3+ years of professional data engineering experience with Spark and cloud data architectures.
• Hands-on experience building pipelines with Databricks, PySpark, and Delta Lake.
• Advanced SQL and Python with strong data modeling skills.
• Experience with Structured Streaming and ingestion tools.
• Experience with Databricks Workflows or Airflow and Azure Data Factory.
• Experience migrating legacy ETL to modern Lakehouse architectures.
• Familiarity with Unity Catalog, data governance, and PII handling.
• Experience with dbt or equivalent transformation framework.

Responsabilidades

• Design, build, and operate batch and streaming data pipelines on Databricks.
• Model and maintain a medallion bronze/silver/gold architecture for analytics and ML.
• Migrate legacy ETL workloads to the Lakehouse with data parity.
• Use Claude or Github Copilot to accelerate development, write tests, and document solutions.
• Write clean Python and SQL, ensure high-quality code through reviews.
• Optimize Spark jobs and Delta tables for performance and cost.
• Implement data quality, lineage, and governance controls with Unity Catalog.
• Debug and resolve production pipeline issues, collaborating with DevOps and analytics teams.

Conocimientos

3+ years of data engineering

Apache Spark

PySpark

Delta Lake

Databricks

SQL

Python

Structured Streaming

Data modeling

Unity Catalog

dbt

Airflow / Databricks Workflows

Herramientas

Databricks

Apache Spark (PySpark)

Delta Lake

Airflow

Azure Data Factory

dbt

Terraform

Descripción del empleo

AgileEngine, LLC. is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse.

You will build batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers. Join a team leveraging AI tools like Claude and GitHub Copilot to accelerate development while ensuring data quality and governance.