Senior Azure Databricks Engineer

Lumenalta

United States

Remote

USD 80,000 - 150,000

Full time

1 hour ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Health insurance
401K Contribution
Professional development fund
15 days paid time off

Job summary

Lumenalta is seeking a Senior Azure Databricks Engineer to lead the design and delivery of enterprise data platforms for clients. You will own the architecture for lakehouse platforms on Databricks and Azure, from governance to pipelines, and set standards for the team.

This hands-on role requires ownership of what gets built and strong communication with client stakeholders. You will lead platform architecture, governance, and scalable pipelines using Delta Lake, Lakeflow, and Python, PySpark,

Qualifications

  • 7+ years of professional experience in data engineering, software engineering, or data platform roles, with at least 3 years of hands-on Databricks work in production.

Responsibilities

  • Platform architecture and ownership across Databricks and Azure.

Skills

Python
PySpark
SQL
Automated testing
CI/CD
Git workflows
Architecture design
Leadership
English communication

Education

Bachelor's degree in Computer Science or Engineering
Master’s degree (preferred)

Tools

Databricks
Delta Lake
Unity Catalog
Lakeflow
Databricks Asset Bundles
Terraform
Azure
ADLS Gen2
Microsoft Entra ID
Key Vault

Job description

At Lumenalta, we partner with forward-thinking organizations to build technology solutions that scale, delight users, and accelerate business growth. Our global teams bring curiosity, commitment, and technical excellence to every project. We value transparency, autonomy, and impact, empowering every team member to do their best work.

About the Role

We are looking for a Senior Azure Databricks Engineer to lead the design and delivery of modern enterprise data platforms for our clients. In this role, you will own the architecture and implementation of lakehouse platforms on Databricks and Azure, from governance and infrastructure to pipelines and production operations. You will set the technical direction for the platform, establish the standards the rest of the team builds on, and serve as the trusted technical voice for client stakeholders. This is a hands-on role for an engineer who brings software engineering discipline to data platform work and who wants real ownership of what gets built.

What You’ll Be Doing
  • Platform architecture and ownership. Lead end-to-end architecture for enterprise lakehouse platforms on Databricks and Azure, owning design decisions across storage, compute, governance, security, and cost, and documenting them so they hold up as the platform grows.
  • Governance and security. Design and implement Unity Catalog strategies, including catalog and schema structure, access control models, lineage and data sharing, aligned with client security and compliance requirements.
  • Pipeline engineering. Architect and build scalable batch and streaming pipelines in Python, PySpark, and SQL using Delta Lake and Lakeflow, including change data capture and medallion layering.
  • Engineering standards. Define and enforce best practices for testing with PyTest, code review, Git based workflows, and CI/CD, and package platform assets with Databricks Asset Bundles so deployments are repeatable and auditable.
  • Infrastructure as code. Own Terraform-based provisioning of Databricks workspaces and Azure resources, including ADLS Gen2, Microsoft Entra ID, and Key Vault, so every environment is reproducible from code.
  • Performance and cost. Tune distributed workloads, choose the right compute model for each workload, and put cost monitoring and controls in place.
  • Technical leadership. Mentor engineers, review architecture and code, and raise the technical bar of the delivery team.
  • Client partnership. Act as the senior technical point of contact for client stakeholders, translating business goals into platform roadmaps, presenting architecture recommendations, and communicating risks and trade-offs clearly.
What We’re Looking For
  • 7+ years of professional experience in data engineering, software engineering, or data platform roles, with at least 3 years of hands-on Databricks work in production.
  • Demonstrated experience as a solution architect, technical lead, or data platform owner on one or more enterprise Databricks implementations.
  • Expert-level Python, PySpark, and SQL, with a consistent practice of automated testing using PyTest.
  • Deep, hands‑on experience with Delta Lake, Unity Catalog, Databricks Asset Bundles, and Lakeflow Jobs.
  • Production experience with Terraform for provisioning Azure and Databricks infrastructure.
  • Strong Azure experience, particularly ADLS Gen2, Microsoft Entra ID, and Azure networking, including secure access patterns between Azure and Databricks.
  • Proven experience building and maintaining CI/CD pipelines and Git based development workflows for data platforms.
  • Advanced English and the ability to communicate technical decisions clearly to engineers and executive stakeholders in a distributed, remote team.
  • Based in a location that allows full overlap with US business hours.
Nice to Have
  • Experience designing lakehouse architectures at enterprise scale, including multi-workspace and multi-environment setups.
  • A track record of optimizing distributed workloads for performance and cost, including serverless compute and liquid clustering.
  • Consulting or client-facing delivery experience, ideally in finance, aerospace, e-commerce, or logistics.
  • Experience implementing data governance, security, and lineage frameworks in regulated environments.
  • Hands‑on experience with Azure Data Factory, including pipelines, triggers, and integration runtimes, or migrating Data Factory workloads to Databricks.
  • Databricks Certified Data Engineer Professional, and ideally additional certifications such as Microsoft Certified: Azure Solutions Architect Expert.
  • A degree in Computer Science, Engineering, or a related field; a Master’s degree is a plus.
Why Lumenalta is an amazing place to work at

At Lumenalta, you can expect that you will:

  • Be 100% dedicated to one project at a time so that you can innovate and grow.
  • Be a part of a team of talented and friendly senior-level developers.
  • Work on projects that allow you to use leading tech.

Salary range: $80,000 - $150,000 annually, with final compensation determined by your qualifications, expertise, experience, and the role’s scope.

Location

This is a fully remote position; however, candidates must be based in regions that align with US time zones to ensure effective collaboration with client and team schedules.

In addition to competitive pay, we offer a variety of benefits to support your professional and personal growth, including:

  • Flexible working hours in a remote environment.
  • Health insurance (medical and dental) for W2 Employees.
  • 401K Contribution.
  • A professional development fund to enhance your skills and knowledge.
  • 15 days of paid time off annually.
  • Access to soft-skill development courses to further your career.

This is a full-time position requiring a minimum of 40 hours per week, Monday through Friday.

At Lumenalta, we are committed to creating an environment that prioritizes growth, work-life balance, and the diverse needs of our team members.

Application Deadline

Applications will be accepted until November 22nd, 2026. Candidates can expect feedback by November 30th, 2026.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Architect - Databricks Migration
Data Architect - Databricks Migration

Lumenalta • United States

Remote
USD 80,000 - 120,000
Health insurance
401K Contribution
Paid time off
+1
Data Engineer - Databricks (Remote)
Data Engineer - Databricks (Remote)

Lumenalta • Raleigh (NC)

Remote
USD 60,000 - 110,000
Flexible working hours
Health insurance (medical and dental)
401K Contribution
+3
Solutions Architect - Databricks (Remote)
Solutions Architect - Databricks (Remote)

Lumenalta • Dallas (TX)

Remote
USD 75,000 - 140,000
Flexible working hours
Health insurance
401K Contribution
+3
Solutions Architect - Databricks (Remote)
Solutions Architect - Databricks (Remote)

Lumenalta • Austin (TX)

Remote
USD 75,000 - 160,000
Flexible working hours
Health insurance
401K Contribution
+3
Senior Databricks Engineer
Senior Databricks Engineer

Lumenalta • United States

Remote
USD 80,000 - 150,000
Health insurance (medical)
401K Contribution
Paid time off
Data Solutions Architect / Technical Lead
Data Solutions Architect / Technical Lead

Lumenalta • United States

Remote
USD 75,000 - 160,000
Health insurance
401K Contribution
15 days paid time off annually
+1
Lead Data Engineer - Databricks (Remote)
Lead Data Engineer - Databricks (Remote)

Lumenalta • Raleigh (NC)

Remote
USD 65,000 - 130,000
Flexible working hours
Health insurance (medical and dental)
401K Contribution
+3
Senior Forward Deployed Engineer - Databricks Apps
Senior Forward Deployed Engineer - Databricks Apps

Lumenalta • United States

Remote
USD 170,000 - 250,000
Senior Databricks Data Platform Engineer (Remote)
Senior Databricks Data Platform Engineer (Remote)

Lumenalta • United States

Remote
USD 80,000 - 150,000
Health insurance (medical)
401K Contribution
Paid time off
Senior Databricks Data Engineer, Spreadsheet & Flat-File Sources
Senior Databricks Data Engineer, Spreadsheet & Flat-File Sources

Lumenalta • United States

Remote
USD 90,000 - 132,000
Health insurance
401K Contribution
Professional development fund
+1