Senior Databricks Data Engineer - Remote & Flexible Hours

Lumenalta; United States; Remote; CAD 80,000 - 150,000; Full time

Benefits offered by this job

Health insurance

Professional development fund

15 days paid time off annually

Flexible working hours

Job summary

Lumenalta is seeking a Senior Data Engineer - Databricks to design and implement scalable cloud-based data platforms. You will blend software engineering with data architecture to build robust pipelines using PySpark, Python, and SQL, and to ensure governance and testing practices.

You will lead architecture discussions, collaborate with distributed teams, apply CI/CD and IaC, and implement components like Delta Lake, Unity Catalog, and Lakeflow Jobs on AWS.

Qualifications

• Databricks expert with enterprise data platform experience.
• Experience as solution architect or data platform owner.
• Strong Python, PySpark and SQL programming.
• Experience with PyTest, Git, CI/CD and DevOps.
• Terraform and AWS (S3/IAM) experience.
• Delta Lake, Unity Catalog, DABs, Lakeflow experience.
• Excellent communication in distributed teams.

Responsibilities

• Design and implement scalable enterprise data platforms using Databricks and AWS.
• Architect and oversee end-to-end data platform implementations.
• Develop robust data pipelines using Python, PySpark, and SQL.
• Apply strong engineering practices including testing, CI/CD, and version control.
• Implement and manage Databricks platform components including Delta Lake, Unity Catalog, DABs, and Lakeflow Jobs.
• Collaborate with engineering teams to ensure maintainable, scalable, and secure data solutions.
• Implement Infrastructure as Code practices to support repeatable environments.
• Contribute to architectural decisions and guide best practices for the data platform.

Skills

Databricks

Python

PySpark

SQL

Git

CI/CD

DevOps

Terraform

AWS

Tools

Delta Lake

Unity Catalog

Databricks Asset Bundles (DABs)

Lakeflow Jobs

PyTest

Job description

Lumenalta is seeking a Senior Data Engineer - Databricks to design and implement scalable cloud-based data platforms. You will blend software engineering with data architecture to build robust pipelines using PySpark, Python, and SQL, and to ensure governance and testing practices.

You will lead architecture discussions, collaborate with distributed teams, apply CI/CD and IaC, and implement components like Delta Lake, Unity Catalog, and Lakeflow Jobs on AWS.