Azure Databricks Lakehouse Architect

Programmers.io; Minneapolis (MN); On-site; USD 140,000 - 190,000; Full time

Job summary

Programmers.io in Minneapolis seeks an Azure Databricks Architect to design scalable Lakehouse solutions on Azure using Databricks, Delta Lake, PySpark, Spark SQL, ADLS Gen2, ADF, Synapse, and governance practices.

The role focuses on cloud data architecture, pipeline development, performance optimization, and migration from legacy systems across SAS, Hadoop, Teradata, and Oracle. Onsite work is required three days per week.

Qualifications

• Hands-on cloud data architecture and Databricks platform design.
• Experience with Delta Lake and medallion architecture.
• Proficient in PySpark, Spark SQL, and data governance.

Responsibilities

• Design and deliver scalable Lakehouse solutions on Azure with Databricks.
• Build and optimize pipelines using Databricks Workflows and Delta Live Tables.
• Work with ADLS Gen2, Synapse, PySpark, Spark SQL for enterprise data needs.
• Optimize Spark workloads through partitioning, sizing, and data skew handling.
• Lead migrations from legacy platforms like SAS, Hadoop, Oracle, and SQL Server.
• Define data modeling, governance, security, lineage, and access controls.

Skills

Databricks

Delta Lake

PySpark

Spark SQL

Data governance

Security

Data modeling

Migration

Cloud architecture

Azure Data Factory

Synapse

ADLS Gen2

Tools

Azure Data Factory

Azure Data Lake Storage Gen2

Databricks SQL

Synapse

Job description

Programmers.io in Minneapolis seeks an Azure Databricks Architect to design scalable Lakehouse solutions on Azure using Databricks, Delta Lake, PySpark, Spark SQL, ADLS Gen2, ADF, Synapse, and governance practices.

The role focuses on cloud data architecture, pipeline development, performance optimization, and migration from legacy systems across SAS, Hadoop, Teradata, and Oracle. Onsite work is required three days per week.