Remote Senior Databricks Platform Architect

Lumenalta; United States; Remote; USD 80,000 - 150,000; Full time

Benefits offered by this job

Health insurance (medical and dental)

401K Contribution

Professional development fund

Flexible remote hours

15 days paid time off

Job summary

Lumenalta seeks a Senior Data Engineer - Databricks to design and implement modern enterprise data platforms in the cloud, blending software engineering with data architecture.

You will build scalable PySpark, Python, and SQL pipelines and lead CI/CD, testing, and governance across the data platform.

Experience with Delta Lake, Unity Catalog, Databricks Asset Bundles, Lakeflow, Terraform, and AWS will help you drive secure, scalable solutions for US clients in a remote-ready setting.

Qualifications

• Deep expertise in Databricks, including platform architecture and best practices.
• Experience as a Solution Architect or Data Platform Owner designing end-to-end implementations.
• Strong programming experience with Python, PySpark, and SQL.
• Experience using testing frameworks such as PyTest.
• Git-based workflows, CI/CD pipelines, and DevOps practices.
• Hands-on experience with Delta Lake, Unity Catalog, Databricks Asset Bundles, and Lakeflow Jobs.
• Infrastructure as Code using Terraform.
• Strong AWS experience, particularly S3 and IAM roles.

Responsibilities

• Design and implement scalable enterprise data platforms using Databricks and AWS.
• Architect and oversee end-to-end data platform implementations.
• Develop robust data pipelines using Python, PySpark, and SQL.
• Apply strong engineering practices including testing, CI/CD, and version control.
• Implement and manage Databricks platform components including Delta Lake, Unity Catalog, Databricks Asset Bundles, and Lakeflow Jobs.
• Collaborate with engineering teams to ensure maintainable, scalable data solutions.
• Implement Infrastructure as Code practices to support repeatable environments.
• Contribute to architectural decisions and guide best practices for the data platform.

Skills

Databricks

Python

PySpark

SQL

CI/CD

DevOps

Git workflows

Terraform

AWS

Delta Lake

Unity Catalog

Databricks Asset Bundles

Lakeflow Jobs

PyTest

Tools

Terraform

Git

Databricks

Job description

Lumenalta seeks a Senior Data Engineer - Databricks to design and implement modern enterprise data platforms in the cloud, blending software engineering with data architecture.

You will build scalable PySpark, Python, and SQL pipelines and lead CI/CD, testing, and governance across the data platform.

Experience with Delta Lake, Unity Catalog, Databricks Asset Bundles, Lakeflow, Terraform, and AWS will help you drive secure, scalable solutions for US clients in a remote-ready setting.