Databricks Platform Architect - Azure Lakehouse Governance

BlueFlag LLC; Dallas (TX); On-site; USD 140,000 - 210,000; Full time

Benefits offered by this job

Competitive salary

Generous annual leave

Comprehensive health plan

401(k) with company match

Life insurance

Job summary

BlueFlag LLC is hiring Databricks Solutions Engineers to standardize and govern a premier data/AI platform for the VA. You will design the platform foundation on Azure, implement governance with Unity Catalog, and build reference pipelines and MLOps patterns.

You’ll train teams and drive adoption across many groups. Ideal candidates bring 12+ years in data engineering, 5+ years on cloud data platforms, and hands-on Databricks experience, with strong Spark and Terraform expertise.

Qualifications

• 12+ years in data engineering or analytics.
• 5+ years on cloud data platforms (Azure preferred).
• 3+ years hands-on with Databricks and Delta Lake.
• Proven Unity Catalog implementation with granular governance.
• Strong Spark skills (PySpark, Spark SQL) and Structured Streaming.

Responsibilities

• Design the Databricks platform foundation in Azure Commercial: workspace topology, VNet injection, Private Link, secure connectivity to ADLS Gen2, Azure SQL, and Event Hubs, provisioned with Terraform.
• Set Unity Catalog governance: catalog and schema design, data classification, row- and column-level security, attribute-based access, lineage, and Delta Sharing.
• Define lakehouse standards for medallion data flow, table layout, and performance (Photon, serverless).
• Build reference pipelines for batch, streaming, and CDC with Lakeflow Declarative Pipelines, Lakeflow Jobs, Auto Loader, and Structured Streaming.
• Set MLOps patterns with MLflow, Unity Catalog model registry and feature tables, and model serving.
• Set guardrails for a platform used by thousands of teams: cluster and budget policies, tagging and chargeback, monitoring, alerting, and audit logging.

Skills

Spark skills

Customer-facing

Communication

Education

Bachelor's degree in information technology or related field

Tools

Terraform

Azure DevOps

GitHub Actions

Databricks

Delta Lake

Job description

BlueFlag LLC is hiring Databricks Solutions Engineers to standardize and govern a premier data/AI platform for the VA. You will design the platform foundation on Azure, implement governance with Unity Catalog, and build reference pipelines and MLOps patterns.

You’ll train teams and drive adoption across many groups. Ideal candidates bring 12+ years in data engineering, 5+ years on cloud data platforms, and hands-on Databricks experience, with strong Spark and Terraform expertise.