Senior Databricks Platform Architect (Azure)

Blueflag-LLP; Raleigh (NC); On-site; USD 150,000 - 190,000; Full time

Benefits offered by this job

Competitive salary

Generous annual leave and holidays

Health and dental plans

401(k) with company match

Life insurance and AD&D

Job summary

BlueFlag is seeking a Databricks Solutions Engineer to design and govern a premier data/AI platform for the Department of Veterans Affairs. You’ll define the Databricks foundation on Azure, build reference pipelines, and set governance across thousands of teams.

You’ll lead MLOps efforts with MLflow, implement guardrails, and develop reusable templates, workshops, and playbooks to accelerate adoption. U.S. work authorization required, strong Azure Databricks skills essential.

Qualifications

• Proven production Databricks experience and platform governance
• Experience designing lakehouse architectures on Azure
• Strong knowledge of Unity Catalog governance and data security
• Hands-on with Delta Lake, OPTIMIZE, VACUUM, and performance tuning
• Familiarity with MLflow, feature stores, and model serving
• Terraform-based infrastructure for Databricks and Azure services
• CI/CD pipelines with Azure DevOps or GitHub Actions
• Experience with healthcare or government cloud environments is a plus

Responsibilities

• Design the Databricks platform foundation in Azure Commercial with Terraform
• Set Unity Catalog governance, data classification, security, and lineage
• Define lakehouse standards for data flow, table layout, and performance
• Build reference pipelines for batch, streaming, and CDC using Lakeflow and Auto Loader
• Establish MLOps patterns with MLflow and feature tables
• Implement guardrails: policies, tagging, chargeback, monitoring, and audit logging
• Create reusable templates and run workshops to drive adoption
• Collaborate with security, networking, compliance, and VA stakeholders

Skills

Databricks

Azure

Unity Catalog

Spark

Structured Streaming

Terraform

CI/CD

Azure DevOps

GitHub Actions

MLflow

Feature stores

Lakehouse

Security governance

Education

Bachelor's degree in information technology or related field

Tools

Terraform

Azure DevOps

GitHub Actions

Job description

BlueFlag is seeking a Databricks Solutions Engineer to design and govern a premier data/AI platform for the Department of Veterans Affairs. You’ll define the Databricks foundation on Azure, build reference pipelines, and set governance across thousands of teams.

You’ll lead MLOps efforts with MLflow, implement guardrails, and develop reusable templates, workshops, and playbooks to accelerate adoption. U.S. work authorization required, strong Azure Databricks skills essential.