Remote: Azure Databricks Lakehouse Architect

GovCIO; Northern (KY); Hybrid; USD 175,000 - 195,000; Full time

Job summary

GovCIO is hiring a Databricks Solutions Expert to architect and implement Azure Databricks as a governed Lakehouse platform for thousands of analytics teams across the VA Data Modernization initiative.

This role blends platform and security architecture with engineering for data ingestion, transformation and ML lifecycle, delivering a scalable, secure analytics foundation; fully remote within the United States.

Qualifications

• Bachelor’s degree in Information Technology or a related field.
• 12+ years of experience in data engineering/analytics; 5+ years building on cloud data platforms (Azure preferred).
• 3+ years hands-on Azure Databricks (platform + pipelines) and Delta Lake.
• Proven experience setting up Unity Catalog with granular governance (RLS/CLS).
• Deep knowledge of Azure networking (VNets, Private Link, NSGs), identity (Entra ID), Key Vault, ADLS Gen2, Event Hubs, Azure SQL/MI, and Data Factory.
• Strong Spark expertise (PySpark/SQL), Structured Streaming, performance tuning, partitioning and storage optimization.
• Practical Terraform experience for Databricks/Azure resources; CI/CD with Azure DevOps or GitHub Actions.
• Security‑first mindset; track record implementing audit logging, policy‑as‑code, and compliance controls.
• Excellent communication skills; ability to standardize, teach, and influence at enterprise scale.

Responsibilities

• Define platform patterns and build reference implementations for a scalable Lakehouse.
• Set standards for cluster policies, Unity Catalog governance, and CI/CD to enable thousands of teams.
• Drive enablement through templates, notebooks, data products, and office hours for broad adoption.
• Lead architecture and design of the landing zone, networking, and secure connectivity to data sources.
• Collaborate with Security, Compliance, and FinOps to codify enterprise standards.

Skills

Enterprise architecture

Azure Databricks

Data engineering

Spark (PySpark/SQL)

CI/CD

Security & governance

Communication

Azure networking

Terraform

Cloud platform design

Education

Bachelor's degree in IT or related field

Tools

Azure Databricks

Delta Lake

Terraform

Azure DevOps / GitHub Actions

Job description

GovCIO is hiring a Databricks Solutions Expert to architect and implement Azure Databricks as a governed Lakehouse platform for thousands of analytics teams across the VA Data Modernization initiative.

This role blends platform and security architecture with engineering for data ingestion, transformation and ML lifecycle, delivering a scalable, secure analytics foundation; fully remote within the United States.