Senior Databricks Lakehouse Migration Lead

PLANIT Group; Reston (VA); Hybrid; USD 130,000 - 170,000; Full time

Job summary

PLANIT Group seeks a Senior Databricks Migration Engineer to lead a government-focused migration project in Reston, VA, with flex to remote work. The role requires transitioning from SQL Server/ADF to Databricks Lakehouse, implementing Delta Lake, DLT, and rigorous governance while enabling Power BI performance.

You will guide a cross-functional team, design scalable data architectures, and deliver comprehensive documentation and knowledge transfer to ensure long-term operational

Qualifications

• Bachelor’s degree or higher in Computer Science, Engineering, or related technical field
• 5+ years in data engineering, data systems, or related roles
• 5+ years on cloud platforms (Azure, AWS, GCP)
• 1+ year leading complex, cross-functional data projects and teams

Responsibilities

• Lead migration from legacy SQL Server and ADF to Databricks Lakehouse (Delta Lake) with best-practice design
• Translate relational warehousing to scalable Lakehouse architectures (Bronze/Silver/Gold)
• Design ETL/ELT frameworks using PySpark, DLT, and Databricks Workflows
• Refine Gold Layer for Power BI performance (dimensional models, star schemas)
• Optimize Databricks SQL Warehouses for high-concurrency Power BI queries
• Implement advanced optimization: Z-Ordering, data skipping, liquid clustering, materialized views
• Enforce governance: cluster sizing, auto-scaling, serverless SQL compute
• Create monitoring dashboards to track DBU consumption and costs
• Establish partitioning strategies and Delta Lake file-size management
• Design data security model with Unity Catalog and access controls
• Align with Azure AD/Microsoft Entra ID and RBAC standards
• Lead pair-programming sessions and code reviews to shift from SQL-centric to Spark-centric
• Produce architecture diagrams, patterns, and optimization playbooks
• Deliver knowledge transfer to ensure self-sufficiency post-migration
• Communicate clearly with technical and non-technical audiences
• Maintain organized, detail-oriented workflow

Skills

Databricks Lakehouse

PySpark

Delta Live Tables

Databricks Workflows

Power BI

Delta Lake

RBAC

Azure Active Directory

Security governance

DBU monitoring

Partition strategies

Unity Catalog

Education

Bachelor's degree in Computer Science / Engineering or related field

Tools

Databricks

Delta Lake

Delta Live Tables (DLT)

Power BI

Job description

PLANIT Group seeks a Senior Databricks Migration Engineer to lead a government-focused migration project in Reston, VA, with flex to remote work. The role requires transitioning from SQL Server/ADF to Databricks Lakehouse, implementing Delta Lake, DLT, and rigorous governance while enabling Power BI performance.

You will guide a cross-functional team, design scalable data architectures, and deliver comprehensive documentation and knowledge transfer to ensure long-term operational