Senior Databricks Data Engineer | Remote Growth & Impact

AgileEngine; Monterrey; Presencial; MXN 900.000 - 1.300.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth opportunities

Competitive compensation

Remote work

Modern projects

Collaborative culture

Well-being programs

Descripción de la vacante

AgileEngine in Monterrey is seeking an experienced Senior Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers.

This role also uses AI tools like Claude and GitHub Copilot to speed up development. You will collaborate with DevOps and analytics teams, implement data governance, and mentor junior engineers while

Formación

• 4+ years of professional experience in data engineering with Spark and cloud data architectures.
• Strong hands-on experience with Databricks, Spark (PySpark), and Delta Lake.
• Advanced SQL and Python with data modeling skills.
• Experience with streaming ingestion (Structured Streaming, Auto Loader, Kafka, Event Hubs).
• Experience with workflow orchestration (Databricks Workflows, Airflow, Azure Data Factory).
• Experience with legacy migrations, data hygiene, or ETL modernization.
• Strong problem-solving, collaboration, and mentoring skills.
• Familiarity with Unity Catalog, data governance, access control, and PII handling.
• Experience with dbt or similar transformation framework.
• Knowledge of secure coding standards and security best practices.
• Experience delivering production data platforms at scale.
• Upper-intermediate English level.

Responsabilidades

• Design, build, and operate batch and streaming data pipelines on Databricks with PySpark and Delta Lake.
• Model and maintain a medallion architecture for analytics, reporting, and ML consumers.
• Migrate legacy ETL workloads to Lakehouse with validated data parity.
• Leverage Claude or Copilot to accelerate development, tests, docs, and prototyping.
• Write clean Python and SQL; ensure quality through reviews and documentation.
• Optimize Spark jobs and Delta tables for performance and cost.
• Implement data quality, lineage, and governance with Unity Catalog.
• Debug, troubleshoot, and resolve production pipeline incidents.
• Participate in Agile or product-centric delivery practices.
• Collaborate with DevOps, platform, and analytics engineers on observability, security, and compliance.

Conocimientos

Databricks

Apache Spark (PySpark)

Delta Lake

SQL

Python

Structured Streaming

Kafka

Event Hubs

Databricks Workflows

Airflow

Azure Data Factory

Unity Catalog

dbt

Security best practices

Data governance

PII handling

Herramientas

Terraform

Azure DevOps

PostgreSQL

Dynatrace

CloudWatch

Databricks system tables

Descripción del empleo

AgileEngine in Monterrey is seeking an experienced Senior Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers.

This role also uses AI tools like Claude and GitHub Copilot to speed up development. You will collaborate with DevOps and analytics teams, implement data governance, and mentor junior engineers while