Azure Databricks Architect

ZipStaff Inc.; Houston (TX); On-site; USD 150,000 - 190,000; Full time

Job summary

ZipStaff is seeking an experienced Azure Databricks Architect to lead the design, development, and optimization of enterprise-scale data platforms within an Oil & Gas context. The role requires hands-on expertise and collaboration with business stakeholders to deliver governed analytics.

You will mentor engineers, drive CI/CD for data pipelines, and optimize Spark workloads on Azure Databricks. On-site in Houston, you will partner with a major energy organization to ensure secure, scalable

Qualifications

• Ten years or more in data engineering or data platform delivery.
• Hands-on Azure Databricks and enterprise-scale implementations.
• Proficiency in PySpark, Python, SQL/T-SQL, and Azure data tools.
• Oil & Gas industry experience required; on-site Houston, TX.

Responsibilities

• Lead architecture and development of scalable data pipelines using Azure Databricks and PySpark.
• Design and optimize data ingestion, transformation, and consumption layers.
• Architect data models, data dictionaries, and reusable data products.
• Implement data governance, security, and compliance standards.
• Tune Spark workloads, Databricks jobs, and platform performance.
• Collaborate with business users and cross-functional teams to translate requirements into solutions.
• Mentor and guide data engineers and junior team members.
• Drive CI/CD automation and DevOps practices for data platforms.
• Support Data Lakehouse architecture and enterprise analytics initiatives.

Skills

Azure Databricks

PySpark

Python

SQL

Data engineering

CI/CD

DevOps

Data governance

Oil & Gas domain

Tools

Azure Data Factory

ADLS

Azure Synapse Analytics

Delta Lake

Job description

Job Title: Azure Databricks Architect

Location: Houston, TX (On-site)

Employment Type: Contract (W2 through ZipStaff)

About the Opportunity

ZipStaff is seeking an experienced Azure Databricks Architect with Oil & Gas domain experience to lead the design, development, and optimization of enterprise-scale data platforms and analytics solutions.

This role sits within a major energy organization’s Azure-based data analytics ecosystem. You will drive data engineering best practices, architect scalable Lakehouse solutions, mentor team members, and partner with business stakeholders to deliver high-performance, secure, and governed data platforms.

What You’ll Do:

• Lead the architecture and development of scalable data pipelines and ETL/ELT frameworks using Azure Databricks and PySpark.
• Design and optimize data ingestion, transformation, and consumption layers.
• Architect data models, data dictionaries, and reusable data products.
• Implement data governance, quality, security, and compliance standards.
• Optimize Spark workloads, Databricks jobs, and overall platform performance.
• Collaborate with business users, data scientists, and cross-functional teams to translate requirements into technical solutions.
• Mentor and guide data engineers and junior team members.
• Drive CI/CD automation and DevOps best practices for data platforms.
• Support Data Lakehouse architecture and enterprise analytics initiatives.

Required Qualifications:

• 10+ years of experience in Data Engineering or Data Platform delivery.
• Strong hands‑on expertise with Azure Databricks and enterprise-scale implementations.
• Proficiency in PySpark / Apache Spark, Python, and SQL / T‑SQL.
• Experience with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure Synapse Analytics, and Delta Lake.
• Solid experience with data modeling, data warehousing, and performance tuning.
• Experience driving CI/CD, Azure DevOps, and automation frameworks for data platforms.
• Strong background in data governance, security, and compliance.
• Proven ability to lead Azure-based cloud data transformation programs.
• Experience mentoring teams and establishing technical standards.
• Oil & Gas industry experience required.
• Ability to work on-site in Houston, TX.

Preferred Qualifications:

• Experience with Unity Catalog.
• Exposure to Airflow, Power BI, or DataOps/MLOps practices.
• Strong stakeholder management and client-facing communication skills.
• Deep experience with Lakehouse architecture.

About ZipStaff

ZipStaff partners with leading organizations to connect skilled data and cloud professionals with high-impact contract opportunities. We focus on quality matches and long-term success.