Databricks Lakehouse Architect | Azure Data Platform Lead

Connect Tech+Talent; United States; On-site; USD 150,000 - 210,000; Full time

Job summary

Connect Tech+Talent seeks an experienced Databricks Architect to design, develop, and implement scalable data solutions on the Databricks Lakehouse Platform for a premier pharmaceutical sourcing and logistics enterprise. The ideal candidate will lead enterprise-level data transformation initiatives, with strong expertise in data architecture, cloud data engineering, data modeling, and governance.

You will collaborate跨跨 with data engineers, data scientists, and business stakeholders to drive

Qualifications

• Bachelor's or Master's in CS, IT, or related field.
• Strong experience with Databricks, Spark, PySpark.
• Expertise in Delta Lake and Medallion Architecture.
• Hands-on experience designing and implementing enterprise data platforms on Azure.
• Strong knowledge of data warehousing, ETL/ELT, data modeling.
• Experience with Unity Catalog, data governance, security, and access management.
• Proficiency in Python and SQL.
• Experience with CI/CD, Git, and infrastructure-as-code practices for data platforms.
• Excellent communication and technical leadership skills.

Responsibilities

• Design and implement scalable data architectures using the Databricks Lakehouse Platform.
• Develop and optimize data pipelines using PySpark, Spark SQL, and Delta Lake.
• Architect and implement Medallion Architecture for enterprise data processing.
• Integrate Databricks with Azure cloud platforms.
• Design and implement batch and streaming data processing solutions.
• Establish data governance, security, access controls, and compliance standards using Unity Catalog and related tools.
• Optimize Spark workloads, cluster configurations, query performance, and overall platform costs.
• Define data modeling, data integration, and data management strategies to support business requirements.
• Collaborate with data engineers, data scientists, business stakeholders, and enterprise architects.
• Provide technical leadership, architecture guidance, code reviews, and best practices for development teams.
• Evaluate emerging data technologies and recommend improvements to the organization's data platform.

Skills

Databricks Platform

Apache Spark

PySpark

Spark SQL

Delta Lake

Lakehouse architecture

Medallion Architecture

Azure

Data modeling

Data governance

Python

SQL

CI/CD

Infrastructure as Code

Leadership

Communication

Education

Bachelor's or Master's in CS/IT/Data Engineering

Tools

Databricks

Azure

Delta Lake

Spark SQL

Unity Catalog

Git

Terraform

Job description

Connect Tech+Talent seeks an experienced Databricks Architect to design, develop, and implement scalable data solutions on the Databricks Lakehouse Platform for a premier pharmaceutical sourcing and logistics enterprise. The ideal candidate will lead enterprise-level data transformation initiatives, with strong expertise in data architecture, cloud data engineering, data modeling, and governance.

You will collaborate跨跨 with data engineers, data scientists, and business stakeholders to drive