Remote Senior Databricks Engineer (Data Platform Architect)

Lumenalta; United States; Remote; USD 80,000 - 150,000; Full time

Benefits offered by this job

Health insurance (medical and dental)

401K Contribution

Professional development fund

15 days of paid time off annually

Access to soft-skill development课程

Flexible working hours in a remote env

Job summary

Lumenalta seeks a Senior Data Engineer - Databricks to design and build modern, scalable data platforms in the cloud. You will work across architecture, data pipelines, and governance, applying best practices from development to deployment.

The ideal candidate has deep Databricks experience and strong Python/PySpark/SQL skills, plus hands-on experience with Delta Lake, Unity Catalog, and IaC with Terraform. Remote work in US time zones is supported.

Qualifications

• Deep expertise in Databricks, including platform architecture and best practices.
• Experience as a Solution Architect or Data Platform Owner designing end-to-end implementations.
• Strong programming experience with Python, PySpark, and SQL.
• Experience using testing frameworks such as PyTest.
• Solid experience with Git-based workflows, CI/CD pipelines, and DevOps practices.
• Hands-on experience with Delta Lake, Unity Catalog, Databricks Asset Bundles (DABs), and Lakeflow Jobs.
• Experience with Infrastructure as Code using Terraform.
• Strong AWS experience, particularly S3 and IAM roles.
• Strong communication and collaboration skills in distributed teams.

Responsibilities

• Design and implement scalable enterprise data platforms using Databricks and AWS.
• Architect and oversee end-to-end data platform implementations.
• Develop robust data pipelines using Python, PySpark, and SQL.
• Apply strong engineering practices including testing, CI/CD, and version control.
• Implement and manage Databricks platform components including Delta Lake, Unity Catalog, Databricks Asset Bundles, and Lakeflow Jobs.
• Collaborate with engineering teams to ensure maintainable, scalable, and secure data solutions.
• Implement Infrastructure as Code practices to support repeatable environments.
• Contribute to architectural decisions and guide best practices for the data platform.

Skills

Databricks

Python

PySpark

SQL

CI/CD

DevOps

Terraform

AWS

Delta Lake

Unity Catalog

DABs

Lakeflow Jobs

Git workflows

PyTest

Tools

Databricks platform

Delta Lake

Unity Catalog

Databricks Asset Bundles

Lakeflow

Terraform

Job description

Lumenalta seeks a Senior Data Engineer - Databricks to design and build modern, scalable data platforms in the cloud. You will work across architecture, data pipelines, and governance, applying best practices from development to deployment.

The ideal candidate has deep Databricks experience and strong Python/PySpark/SQL skills, plus hands-on experience with Delta Lake, Unity Catalog, and IaC with Terraform. Remote work in US time zones is supported.