Databricks Solutions Expert (Remote)

GovCIO; Carson City (NV); Remote; USD 175,000 - 195,000; Full time

Benefits offered by this job

Employee Assistance Program (EAP)

Corporate Discounts

Learning & Development platform

Training, Education and Certification

Referral Bonus Program

Internal Mobility Program

Pet Insurance

Flexible Work Environment

Job summary

GovCIO is hiring a Databricks Solutions Expert to architect and implement Azure Databricks as a multi-tenant, governed lakehouse for VA Data Modernization. You will lead platform design, security, and CI/CD enablement for thousands of analytics teams across the enterprise.

The role blends engineering with platform governance, supporting secure, scalable analytics in a fully remote position within the United States. Prior Azure Databricks and Delta Lake experience is required.

Qualifications

• Bachelor’s degree in Information Technology or equivalent.
• 12+ years in data engineering/analytics; 5+ years on cloud data platforms (Azure preferred).
• 3+ years hands-on Azure Databricks (platform + pipelines) and Delta Lake.
• Experience setting up Unity Catalog with granular governance (RLS/CLS).
• Strong knowledge of Azure networking, Entra ID, Key Vault, ADLS Gen2, Event Hubs, Azure SQL/MI, Data Factory.
• Strong Spark expertise (PySpark/SQL), Structured Streaming, performance tuning.
• Terraform for Databricks/Azure resources; CI/CD with Azure DevOps or GitHub Actions.
• Security-first mindset; audit logging, policy-as-code, and compliance controls.
• Excellent communication; ability to standardize and influence at enterprise scale.

Responsibilities

• Define platform patterns and reference implementations for Azure Databricks.
• Establish governance via Unity Catalog and multi-region metastores.
• Develop lakehouse foundations with Delta Lake and medallion data flow standards.
• Enable thousands of analytics teams with scalable workspace strategies and guardrails.
• Implement HA/DR and Terraform-based provisioning across environments.

Skills

Azure Databricks

Delta Lake

Unity Catalog

PySpark/SQL

Terraform

Azure DevOps

GitHub Actions

Security & Compliance

Data Governance

Education

Bachelor’s degree in IT or related field

Tools

Terraform

Azure DevOps

GitHub Actions

dbx

Job description

United States

Suitability/Public Trust

Fully remote

Information Technology

Overview

GovCIO is hiring a Databricks Solutions Expert to support the Department of Veterans Affairs (VA) Data Modernization initiative who will architect, design, and implement Azure Databricks as a multi-tenant, governed Lakehouse platform that can be adopted by thousands of analytics teams across the enterprise. This role blends architecture (platform, security, governance, networking) with engineering (data ingestion, transformation, streaming, ML lifecycle) to deliver a secure, scalable, cost‑efficient analytics foundation on Azure. This position is fully remote within the United States.

Responsibilities

The Databricks Solutions Expert will will define platform patterns, build reference implementations, set standards (cluster policies, Unity Catalog governance, CI/CD), and drive enablement so product, analytics, and data science teams can deliver reliable, compliant, and performant insights at scale.

Platform Architecture & Design

• Blueprint the Azure Databricks landing zone: workspace topology (prod/non-prod), network architecture (VNet injection, Private Link, NAT), secure connectivity to ADLS Gen2, Azure SQL/MI, Event Hubs, and other data sources.

• Governance with Unity Catalog: multi-region metastores, catalog/schema/table design, data classification tiers, row/column-level security, data lineage, and cross-domain data sharing patterns (Delta Sharing).

• Lakehouse foundations: Delta Lake storage design (bronze/silver/gold), medallion data flow standards, partitioning, Z‑ordering, OPTIMIZE/VACUUM policies, and performance best practices (Photon, AQE).

• Scalability for “thousands of teams”: multi-workspace strategy, tenancy model (shared vs. dedicated), workspace baselines, cluster policy tiers, and guardrails to prevent noisy-neighbor and cost runaways.

• Reliability: HA/DR strategy, regional deployments, backup/restore, versioning, and repeatable environment provisioning via Terraform.

Security, Compliance & Access Control

• Identity & access: Entra ID (Azure AD) SSO, SCIM user/group provisioning, service principals/managed identities, attribute-based/role-based access controls mapped to Unity Catalog.

• Secrets & credentials: Key Vault–backed secret scopes, credential passthrough patterns, token hygiene (PAT governance).

• Data protection: encryption at rest/in transit, private endpoints, data exfil and egress controls, policy-as-code, and audit logging to Log Analytics or secure storage.

• Compliance: enforce enterprise policies (PII/PHI handling), data retention, legal hold, and regulatory reporting with auditable lineage.

Engineering & Enablement

• Pipelines: build DLT (Delta Live Tables) pipelines and jobs for batch and streaming (Structured Streaming) with CDC (e.g., via Auto Loader) from enterprise sources.

• Performance engineering: optimize notebooks/SQL/ETL (Photon, caching, skew mitigation), tune cluster sizing, and set standards for reliable, fast jobs.

• MLOps: integrate MLflow for experiment tracking, model registry, feature store (Unity Catalog), and serving patterns where appropriate.

• Observability: end-to-end monitoring (jobs, clusters, UC audits), dashboarding, alerting, and SLOs; integrate with Azure Monitor/Log Analytics.

• Enablement: build reusable reference accelerators (templates, example notebooks, data products), run playbooks, and conduct office hours to uplevel 1000s of teams.

DevOps, Automation & Cost Management

• Infrastructure-as-Code: provision workspaces, catalogs, cluster policies, and permissions via Terraform (Databricks provider), with pipelines in Azure DevOps/GitHub.

• CI/CD: notebook/package deployment, testing harnesses (dbx/pytest), environment promotion, and artifact versioning.

• FinOps: cost modeling, budgets/alerts, instance pools, auto-termination, serverless SQL, tagging/chargeback, and usage analytics for executive reporting.

Stakeholder Management & Governance

• Partner with Security, Networking, Compliance, and FinOps to codify enterprise standards.

• Establish a Lakehouse Platform Council to ratify patterns and review exceptions.

• Create adoption metrics, business case narratives, TCO models, and executive updates.

Qualifications

Required Skills and Experience

• Bachelor’s degree in Information Technology or a related field. (or commensurate experience)

• 12+ years of experience in data engineering/analytics; 5+ years building on cloud data platforms (Azure preferred).

• 3+ years hands-on Azure Databricks (platform + pipelines) and Delta Lake.

• Proven experience setting up Unity Catalog with granular governance (RLS/CLS).

• Deep knowledge of Azure networking (VNets, Private Link, NSGs), identity (Entra ID), Key Vault, ADLS Gen2, Event Hubs, Azure SQL/MI, and Data Factory.

• Strong Spark expertise (PySpark/SQL), Structured Streaming, performance tuning, partitioning and storage optimization.

• Practical Terraform experience for Databricks/Azure resources; CI/CD with Azure DevOps or GitHub Actions.

• Security-first mindset; track record implementing audit logging, policy-as-code, and compliance controls.

• Excellent communication skills; ability to standardize, teach, and influence at enterprise scale.

Preferred Skills and Experience

• Experience operating platforms for >500 concurrent users and 1000s of analytics teams.

• Knowledge of Photon, DLT, Workflows, Lakehouse ML (MLflow, feature store), Delta Sharing.

• Exposure to FinOps and chargeback models for data platforms.

• Experience with Synapse/Fabric interoperability, and data virtualization patterns.

• Background in data modeling (medallion, dimensional, domain‑driven design) and data quality (expectations, SLAs).

Certifications

• Databricks: Databricks Certified Data Engineer Professional, Lakehouse Fundamentals, Machine Learning Associate/Professional.

• Microsoft: Azure Solutions Architect Expert (AZ‑305), Azure Data Engineer (DP‑203), Azure Security Engineer (AZ‑500).

• HashiCorp: Terraform Associate.

Technical Stack & Tools

• Core: Azure Databricks (Unity Catalog, Delta Lake, DLT, Workflows, Photon), ADLS Gen2, Azure Key Vault, Entra ID, Private Link.

• Data Integration: Auto Loader, ADF/Synapse pipelines, Event Hubs/Kafka, JDBC/ODBC.

• DevOps/Infra: Terraform (Databricks & Azure providers), Azure DevOps/GitHub Actions, dbx.

• Observability: Databricks audit logs, Azure Monitor, Log Analytics, custom usage analytics.

• Languages: Python (PySpark), SQL, Scala (optional).

• ML: MLflow, Feature Store (Unity Catalog), model serving patterns.

Clearance Required

• Ability to obtain and maintain a suitability/Public Trust

Posted Salary Range

USD $175,000.00 - USD $195,000.00 /Yr.

Company Overview

GovCIO is a team of transformers--people who are passionate about transforming government IT. Every day, we make a positive impact by delivering innovative IT services and solutions that improve how government agencies operate and serve our citizens.

But we can't do it alone. We need great people to help us do great things - for our customers, our culture, and our ability to attract other great people. We are changing the face of government IT and building a workforce that fuels this mission. Are you ready to be a transformer?

What You Can Expect

Interview & Hiring Process

If you are selected to move forward through the process, here’s what you can expect:

• During the Interview Process

• Virtual video interview conducted via video with the hiring manager and/or team

• Camera must be on

• A valid photo ID must be presented during each interview

• During the Hiring Process

• Enhanced Biometrics ID verification screening

• Background check, to include:

• Criminal history (past 7 years)

• Verification of your highest level of education

• Verification of your employment history (past 7 years), based on information provided in your application

Employee Perks

At GovCIO, we consistently hear that meaningful work and a collaborative team environment are two of the top reasons our employees enjoy working here. In addition, our employees have access to a range of perks and benefits to support their personal and professional well-being, beyond the standard company offered health benefits, including:

• Employee Assistance Program (EAP)

• Corporate Discounts

• Learning & Development platform, to include certification preparation content

• Training, Education and Certification Assistance*

• Referral Bonus Program

• Internal Mobility Program

• Pet Insurance

• Flexible Work Environment

*Available to full-time employees

Our employees’ unique talents and contributions are the driving force behind our success in supporting our customers, which ultimately fuels the success of our company. Join us and be a part of a culture that invests in its people and prioritizes continuous enhancement of the employee experience.

We are an Equal Opportunity Employer.

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, disability, or status as a protected veteran. EOE, including disability/vets.

Posted Pay Range

The posted pay range, if referenced, reflects the range expected for this position at the commencement of employment, however, base pay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, education, experience, and internal equity. The total compensation package for this position may also include other compensation elements, to be discussed during the hiring process. If hired, employee will be in an “at-will position” and the GovCIO reserves the right to modify base salary (as well as any other discretionary payment or compensation program) at any time, including for reasons related to individual performance, GovCIO or individual department/team performance, and market factors.