Senior Azure Databricks Engineer

Lumenalta

Ciudad de México

Remote

MXN 2,582,000 - 3,872,000

Full time

31 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Lumenalta is seeking a Senior Azure Databricks Engineer to lead design and delivery of enterprise data platforms on Databricks and Azure. You will own architecture, governance, pipelines, and production operations, while setting technical direction for the team.

This hands-on role requires 7+ years in data/engineering, strong Python/PySpark/SQL, Delta Lake, and Terraform experience. Remote with US hours overlap; option to mentor and shape client roadmaps.

Qualifications

  • 7+ years in data/engineering or data platform roles with 3+ years Databricks in production.
  • Experience as solution architect, tech lead, or data platform owner on enterprise Databricks.
  • Expert-level Python, PySpark, SQL with automated testing using PyTest.
  • Deep Delta Lake, Unity Catalog, Asset Bundles, and Lakeflow experience.
  • Terraform for provisioning Azure and Databricks infrastructure.
  • Strong Azure experience: ADLS Gen2, Entra ID, Azure networking.
  • Proven CI/CD pipelines and Git-based development for data platforms.
  • Advanced English; able to communicate decisions clearly to distributed teams.

Responsibilities

  • Platform architecture and ownership: lead end-to-end architecture for enterprise lakehouse platforms on Databricks and Azure.
  • Governance and security: design Unity Catalog strategies and access models.
  • Pipeline engineering: build scalable batch and streaming pipelines in Python, PySpark, SQL with Delta Lake and Lakeflow.
  • Engineering standards: enforce testing with PyTest, code reviews, Git workflows, and CI/CD.
  • Infrastructure as code: own Terraform provisioning of Databricks workspaces and Azure resources.
  • Performance and cost: tune workloads and implement cost monitoring and controls.
  • Technical leadership: mentor engineers, review architecture and code, raise delivery standards.
  • Client partnership: act as senior technical contact, translate goals into platform roadmaps and communicate risks.

Skills

Python
PySpark
SQL
Databricks
Terraform
CI/CD
Git workflows
PyTest
Advanced English

Education

Bachelor's degree in CS/Engineering
Master's degree (plus)

Tools

Delta Lake
Unity Catalog
Databricks Asset Bundles
Lakeflow
Azure Databricks

Job description

At Lumenalta, we partner with forward-thinking organizations to build technology solutions that scale, delight users, and accelerate business growth. Our global teams bring curiosity, commitment, and technical excellence to every project. We value transparency, autonomy, and impact, empowering every team member to do their best work.

About the Role

We are looking for a Senior Azure Databricks Engineer to lead the design and delivery of modern enterprise data platforms for our clients. In this role, you will own the architecture and implementation of lakehouse platforms on Databricks and Azure, from governance and infrastructure to pipelines and production operations. You will set the technical direction for the platform, establish the standards the rest of the team builds on, and serve as the trusted technical voice for client stakeholders. This is a hands‑on role for an engineer who brings software engineering discipline to data platform work and who wants real ownership of what gets built.

What You'll Be Doing
  • Platform architecture and ownership. Lead end-to-end architecture for enterprise lakehouse platforms on Databricks and Azure, owning design decisions across storage, compute, governance, security, and cost, and documenting them so they hold up as the platform grows.
  • Governance and security. Design and implement Unity Catalog strategies, including catalog and schema structure, access control models, lineage and data sharing, aligned with client security and compliance requirements.
  • Pipeline engineering. Architect and build scalable batch and streaming pipelines in Python, PySpark, and SQL using Delta Lake and Lakeflow, including change data capture and medallion layering.
  • Engineering standards. Define and enforce best practices for testing with PyTest, code review, Git based workflows, and CI/CD, and package platform assets with Databricks Asset Bundles so deployments are repeatable and auditable.
  • Infrastructure as code. Own Terraform-based provisioning of Databricks workspaces and Azure resources, including ADLS Gen2, Microsoft Entra ID, and Key Vault, so every environment is reproducible from code.
  • Performance and cost. Tune distributed workloads, choose the right compute model for each workload, and put cost monitoring and controls in place.
  • Technical leadership. Mentor engineers, review architecture and code, and raise the technical bar of the delivery team.
  • Client partnership. Act as the senior technical point of contact for client stakeholders, translating business goals into platform roadmaps, presenting architecture recommendations, and communicating risks and trade-offs clearly.
What We're Looking For
  • 7+ years of professional experience in data engineering, software engineering, or data platform roles, with at least 3 years of hands‑on Databricks work in production.
  • Demonstrated experience as a solution architect, technical lead, or data platform owner on one or more enterprise Databricks implementations.
  • Expert‑level Python, PySpark, and SQL, with a consistent practice of automated testing using PyTest.
  • Deep, hands‑on experience with Delta Lake, Unity Catalog, Databricks Asset Bundles, and Lakeflow Jobs.
  • Production experience with Terraform for provisioning Azure and Databricks infrastructure.
  • Strong Azure experience, particularly ADLS Gen2, Microsoft Entra ID, and Azure networking, including secure access patterns between Azure and Databricks.
  • Proven experience building and maintaining CI/CD pipelines and Git based development workflows for data platforms.
  • Advanced English and the ability to communicate technical decisions clearly to engineers and executive stakeholders in a distributed, remote team.
  • Based in a location that allows full overlap with US business hours.
Nice to Have
  • Experience designing lakehouse architectures at enterprise scale, including multi-workspace and multi‑environment setups.
  • A track record of optimizing distributed workloads for performance and cost, including serverless compute and liquid clustering.
  • Consulting or client‑facing delivery experience, ideally in finance, aerospace, e‑commerce, or logistics.
  • Experience implementing data governance, security, and lineage frameworks in regulated environments.
  • Hands‑on experience with Azure Data Factory, including pipelines, triggers, and integration runtimes, or migrating Data Factory workloads to Databricks.
  • Databricks Certified Data Engineer Professional, and ideally additional certifications such as Microsoft Certified: Azure Solutions Architect Expert.
  • A degree in Computer Science, Engineering, or a related field; a Master's degree is a plus.
Why Lumenalta is an amazing place to work at

At Lumenalta, you can expect that you will:

  • Be 100% dedicated to one project at a time so that you can innovate and grow.
  • Be a part of a team of talented and friendly senior‑level developers.
  • Work on projects that allow you to use leading tech.
Location

This is a fully remote position; however, candidates must be based in regions that align with the Pacific, Central, or Eastern U.S. time zones to ensure effective collaboration with client and team schedules.

Application Deadline

Applications will be accepted until November 22nd, 2026. Candidates can expect feedback by November 30th, 2026.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Architect - Databricks Migration
Data Architect - Databricks Migration

Lumenalta • Ciudad de México

Remote
MXN 600,000 - 1,200,000
Senior Infrastructure Engineer - Database/ETL Infrastructure
Senior Infrastructure Engineer - Database/ETL Infrastructure

Lumenalta • Ciudad de México

Remote
MXN 2,582,000 - 3,504,000
Senior Azure Databricks Architect - Remote
Senior Azure Databricks Architect - Remote

Lumenalta • Ciudad de México

Remote
MXN 2,582,000 - 3,872,000
Software Engineer Lead
Software Engineer Lead

Lumenalta • Ciudad de México

Remote
MXN 2,582,000 - 3,688,000
Machine Learning Engineer
Machine Learning Engineer

Lumenalta • Ciudad de México

Remote
MXN 1,660,000 - 2,766,000
Cloud Engineer Azure Databricks
Cloud Engineer Azure Databricks

Turtle Trax S.A. • Región Centro

On-site
MXN 1,000,000 - 1,500,000
Remote Data Architect: Databricks Migration Lead
Remote Data Architect: Databricks Migration Lead

Lumenalta • Ciudad de México

Remote
MXN 600,000 - 1,200,000
Senior AI Engineer
Senior AI Engineer

Lumenalta • Ciudad de México

Remote
MXN 1,660,000 - 2,766,000
Senior Azure Databricks and PySpark
Senior Azure Databricks and PySpark

Daikin Manufacturing México • Monterrey

Hybrid
MXN 900,000 - 1,400,000
Remote Data Tech Lead: Architect, Mentor, Hands-On
Remote Data Tech Lead: Architect, Mentor, Hands-On

Lumenalta • Ciudad de México

Hybrid
MXN 2,226,000 - 3,339,000