Databricks Lakehouse Data Engineer

AgileEngine; Mexico; On-site; PHP 7,524,000 - 10,031,000; Full time

Benefits offered by this job

Annual learning budget

Remote work with flexible hours

Competitive compensation

Job summary

AgileEngine is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will design and run batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers.

You’ll optimize for performance and cost, implement data governance with Unity Catalog, and collaborate across teams. This role offers 100% remote work with flexible hours and a strong focus on growth and mentorship.

Qualifications

• 3+ years of professional data engineering experience.
• Hands-on with Apache Spark and cloud data architectures.
• Strong SQL and Python programming skills.
• Experience building batch and streaming pipelines.
• Knowledge of data governance and quality practices.

Responsibilities

• Design, build, and operate batch and streaming data pipelines on Databricks.
• Apply a medallion bronze/silver/gold architecture for analytics and ML.
• Migrate legacy ETL workloads to Lakehouse with validated parity.
• Use Claude or Copilot to accelerate development and testing.
• Write clean Python and SQL and perform code reviews.
• Optimize Spark jobs and Delta tables for cost and performance.
• Implement data quality, lineage, and governance controls.
• Debug and resolve pipeline issues in production.
• Collaborate with DevOps, platform, and analytics engineers on observability and security.

Skills

3+ years experience

Data modeling

Problem solving

Collaboration

Communication

Tools

Databricks

Apache Spark (PySpark)

Delta Lake

Kafka

Event Hubs

Airflow

Azure Data Factory

dbt

Terraform

Job description

AgileEngine is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will design and run batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers.

You’ll optimize for performance and cost, implement data governance with Unity Catalog, and collaborate across teams. This role offers 100% remote work with flexible hours and a strong focus on growth and mentorship.