Databricks Program Lead

truData Solutions; San Antonio (TX); On-site; USD 140,000 - 190,000; Full time

Benefits offered by this job

Equal Opportunity Employer

Job summary

truData Solutions seeks a Databricks: Program Lead to own end-to-end migration of an enterprise data warehouse from Azure Synapse to the Databricks Lakehouse. You will direct multiple concurrent workstreams including platform foundation, pipelines, data reconciliation, and BI cutover, while guiding delivery through Agile ceremonies and stakeholder engagement.

You will shape target Lakehouse architecture, govern Unity Catalog, and lead CI/CD, cost optimization, and performance tuning.

Qualifications

• Experience with Databricks Lakehouse architecture and Unity Catalog governance.
• Proven track record migrating Azure Synapse objects to Databricks (SQL objects to Spark SQL).
• Experience migrating data pipelines using Databricks-native orchestration and Delta Live Tables.
• Ability to define CI/CD and environment promotion standards for Databricks deployments.

Responsibilities

• Own end-to-end delivery of a data warehouse migration project across multiple workstreams.
• Direct platform, pipeline, data migration, and BI cutover activities.
• Lead Agile ceremonies and manage program backlog and epics in Jira.
• Interface with client stakeholders and translate technical risk to business impact.
• Oversee cost/performance guardrails and drive delivery to closure without escalation.
• Lead BI cutover planning and semantic model re-pointing for Power BI.

Tools

Databricks Lakehouse

Unity Catalog governance

Delta Lake design

Cluster sizing

SSIS/ETL migration

Azure Synapse migration

Databricks Workflows

Power BI integration

CI/CD pipelines (Databricks Asset Bundles)

Jira/Confluence

Azure DevOps

Job description

Databricks: Program Lead (Data Warehouse Migration)

The Databricks: Program Lead owns end-to-end delivery of an enterprise data warehouse migration from Azure Synapse Analytics and Azure Data Factory to the Databricks Lakehouse Platform. This role directs multiple concurrent workstreams — platform foundation, pipeline re-engineering, data migration and reconciliation, and BI/consumer cutover — running delivery through Agile ceremonies and tooling (Jira, Confluence, Azure DevOps) while serving as the primary technical and executive interface to client stakeholders.

Technical Competencies (Target: Advanced–Expert)

• Owns the target-state Databricks Lakehouse architecture — Unity Catalog governance, medallion layering, Delta Lake table design, cluster and SQL warehouse sizing, and workspace topology across dev, test, and production. Experienced in full-cycle enterprise databricks implementation preferably in Azure environment.
• Directs the conversion of Azure Synapse dedicated SQL pool objects to Databricks — T‑SQL stored procedures, external tables, and distribution/partitioning strategies re‑engineered into Spark SQL, Delta tables, and Databricks Workflows
• Governs migration of Azure Data Factory pipelines to Databricks‑native orchestration (Workflows, Delta Live Tables, or Lakeflow Declarative Pipelines),
• Establishes the data migration and reconciliation approach — historical load strategy, row and aggregate validation, checksum testing, and dual‑run parallel operation until cutover sign‑off
• Defines CI/CD and environment promotion standards using Databricks Asset Bundles, Git‑based version control, and automated deployment pipelines
• Leads BI and consumer cutover planning for Power BI and downstream applications, including semantic model repointing, performance benchmarking against Synapse baselines, and query optimization
• Sets cost and performance guardrails — DBU consumption forecasting, Photon and serverless adoption decisions, autoscaling policies, and ongoing platform cost optimization
• Reviews and approves technical designs, code, and test evidence produced by engineering pods, resolving escalated architectural decisions
• Leads through influence across client and internal teams — sets delivery cadence, holds pods accountable, and drives decisions to closure without escalation
• Runs Agile delivery discipline in Jira — owns the program backlog and epic structure, enforces sprint planning, refinement, stand‑up, and retrospective rhythm, and keeps burndown, velocity, and dashboards credible
• Communicates with executive presence — translates technical migration risk into business impact for client sponsors and steering committees
• Manages conflict and competing priorities directly, negotiating scope and sequencing with stakeholders rather than absorbing schedule pressure

Business Competencies

• Owns the program plan, budget, and resource model — forecasts burn against budget
• Manages scope formally — identifies change through the Jira intake path, quantifies impact, and drives change orders with the client rather than absorbing creep
• Runs program governance: steering committee cadence, RAID log ownership, status reporting, and stage‑gate sign‑offs for each migration wave

About truData

Headquartered in San Diego, CA truData is a technology services provider and a data‑oriented company specializing in data and analytics and integration offerings delivered through cloud solutions. We partner with clients in their Digital Transformation journey, ensuring improved consumer insight, reduced complexity whilst increasing revenues, and streamlined processes.

Every truData employee is unique. We each have our own diverse experience, and that’s what enables us to innovate and deliver excellence to our customers. At truData you will have a voice, you will be heard, and you will make a difference. Come join us!

At truData we embrace change and diversity. Our culture is based upon inclusivity, because when new perspectives are brought to the table, magic happens. Come join our team and help lets achieve greatness together.

TruData is proud to be an Equal Opportunity Employer