Databricks Data Engineer — Remote Lakehouse & AI Pipelines

AgileEngine, LLC.; Rosarito; Presencial; MXN 700.000 - 1.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Competitive compensation

Remote with flexible hours

Meaningful projects

Collaborative culture

Well-being support

Descripción de la vacante

AgileEngine, LLC. is seeking a Middle Data Engineer to modernize a 15-year-old warehouse into a governed Databricks Lakehouse. You will design batch and streaming pipelines with PySpark and Delta Lake, using a medallion architecture across bronze, silver, and gold layers.

Collaboration with DevOps and analytics teams will ensure observability, security, and governance. The role emphasizes migrating legacy workloads, writing clean Python/SQL, and leveraging AI tools like Claude and GitHub Copilot

Formación

• 3+ years of professional data engineering experience with Spark and cloud data architectures.
• Strong SQL and Python skills; data modeling in dimensional and Lakehouse patterns.
• Experience with Structured Streaming, Auto Loader, Kafka or Event Hubs.
• Experience with Databricks Workflows and data governance concepts.

Responsabilidades

• Design, build, and operate batch and streaming data pipelines on Databricks using PySpark and Delta Lake.
• Maintain a medallion (bronze/silver/gold) architecture for analytics, reporting, and ML.
• Migrate legacy ETL workloads to the Lakehouse with data parity.
• Leverage Claude or Copilot to accelerate development, write tests, and document code.
• Write clean Python and SQL; contribute to code reviews and documentation.
• Optimize Spark jobs and Delta tables for performance and cost.
• Implement data quality, lineage, and governance with Unity Catalog.

Conocimientos

3+ years data engineering

Apache Spark (PySpark)

SQL

Python

Herramientas

Databricks

Delta Lake

Kafka

Event Hubs

Airflow

Azure Data Factory

Unity Catalog

Descripción del empleo

AgileEngine, LLC. is seeking a Middle Data Engineer to modernize a 15-year-old warehouse into a governed Databricks Lakehouse. You will design batch and streaming pipelines with PySpark and Delta Lake, using a medallion architecture across bronze, silver, and gold layers.

Collaboration with DevOps and analytics teams will ensure observability, security, and governance. The role emphasizes migrating legacy workloads, writing clean Python/SQL, and leveraging AI tools like Claude and GitHub Copilot