Director Data Engineering

Straive; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Straive is seeking a Director of Data Engineering to lead end-to-end data engineering client engagements, delivering enterprise-grade Lakehouse and modern data platform solutions on Azure and Databricks.

You will bridge technical depth with executive client management, mentor engineering teams across onshore and offshore, govern delivery margins, and drive architecture, governance, and proactive account growth.

Qualifications

• 14+ years IT/Software Engineering experience; 8+ years data engineering; 4+ years leading multi-project delivery teams.
• Hands-on mastery of the Databricks Lakehouse Platform including Spark internals, PySpark, Delta Lake, Delta Live Tables (DLT), Unity Catalog.
• Deep experience with Azure data stack: ADF, ADLS Gen2, Microsoft Fabric, Key Vault, Event Hub/IoT Hub, Monitor.
• Proven data architecture modeling in Star/Snowflake/Data Vault 2.0 or Lakehouse modeling.
• CI/CD pipelines for data using Azure DevOps, GitHub Actions, Terraform/Bicep, containerized deployments.
• Advanced coding in Python/PySpark and ANSI SQL.
• Experience managing multi-client engagements and steering committee presentations.

Responsibilities

• Own end-to-end delivery across concurrent enterprise data engineering programs from inception to production support.
• Define delivery frameworks, SLAs, milestones, capacity planning, and risk mitigation for on-time, within-budget outcomes.
• Own delivery profitability, utilization metrics, and scope across active accounts.
• Establish quality benchmarks, CI/CD/DataOps practices, and release management across teams.
• Serve as SME and escalation architect for Databricks and Azure platform implementations.
• Lead architecture blueprints for Lakehouse implementations using Medallion Architecture and Delta technologies.
• Oversee cloud migration strategies from on-prem to Azure Databricks.
• Mentor and scale onshore/offshore delivery teams; drive recruiting and competency development.
• Drive alliance initiatives with Microsoft and Databricks.

Skills

Data engineering

Leadership

Cloud platforms

Python/PySpark

SQL

CI/CD

Stakeholder management

Spark internals

Tools

Databricks

Azure

Delta Lake

Unity Catalog

Azure DevOps

GitHub Actions

Terraform

Bicep

Spark

Job description

The Director of Data Engineering will hold end-to-end accountability for leading, managing, and delivering multiple high-impact data engineering client engagements. This role requires an executive leader with deep hands-on expertise across Azure and Databricks who can drive technical architecture, govern engagement delivery, oversee project P&L/margins, and mentor high-performing engineering teams. You will bridge technical depth with executive client management to deliver enterprise-grade Lakehouse and modern data platform solutions.

Key Responsibilities

Multi-Engagement Delivery & Governance

• Own end-to-end delivery accountability across concurrent enterprise data engineering programs from inception, scoping, architecture, and sprint execution through production deployment and hypercare.
• Define delivery frameworks, SLAs, project milestones, resource capacity planning, and risk mitigation strategies to ensure predictable, on-time, and within-budget delivery.
• Own delivery commercials, engagement profitability, utilization metrics, and scope management across all active accounts.
• Establish and monitor quality benchmarks, coding standards, CI/CD/DataOps practices, and release management across teams.

Technical Leadership & Hands-on Architecture

• Serve as the principal SME and escalation architect for complex Databricks and Azure platform implementations.
• Lead architecture blueprints for enterprise Lakehouse implementations utilizing Medallion Architecture (Bronze/Silver/Gold), Delta Lake, Delta Live Tables (DLT), and Unity Catalog.
• Direct large-scale cloud migration strategies from legacy on-prem systems (e.g., Hadoop, Teradata, SQL Server) to Azure Databricks.
• Perform high-level code and architecture reviews, focusing on Spark distributed compute optimization, cluster tuning, I/O bottlenecks, and Azure cloud spend/finops governance.

Client Engagement & Strategic Account Growth

• Act as the trusted technical advisor and primary escalation point for client stakeholders.
• Collaborate with sales and client partners on pre-sales pursuits, high-level estimations, RFPs, technical defense, and statements of work (SOWs).
• Identify organic account expansion opportunities by recommending advanced data platform features, GenAI integration via Databricks Mosaic AI, and modern data governance.

Team Leadership, Hiring & Practice Development

• Manage, mentor, and scale a multi-tiered team of Data Architects, Tech Leads, and Data Engineers across onshore and offshore delivery models.
• Establish career pathways, skill development roadmaps, and competency-building programs around Azure Data Services and Databricks.
• Spearhead hiring by leading technical bar-raiser interviews and establishing rigorous evaluation standards.
• Drive strategic alliance initiatives and partner certifications with Microsoft and Databricks.

Required Qualifications & Experience

• Experience: 14+ years of total experience in IT/Software Engineering, with at least 8+ years focused in Data Engineering and 4+ years leading multi-project delivery teams or data practice groups.
• Databricks Ecosystem: Hands-on mastery of the Databricks Lakehouse Platform, including Apache Spark internals, PySpark, Delta Lake, Delta Live Tables (DLT), Unity Catalog, Databricks SQL, Databricks Workflows, and Lakeflow.
• Azure Cloud Data Stack: Deep practical experience with Azure Data Factory (ADF), ADLS Gen2, Microsoft Fabric, Azure Key Vault, Azure Event Hub/IoT Hub, and Azure Monitor.
• Data Architecture & Modeling: Demonstrated track record architecting enterprise-scale analytical systems using Star Schema, Snowflake Schema, Data Vault 2.0, or Lakehouse dimensional modeling.
• DataOps & CI/CD: Hands-on experience designing robust CI/CD pipelines for data using Azure DevOps, GitHub Actions, Terraform/Bicep, and containerized deployment workflows.
• Programming: Advanced coding proficiency in Python/PySpark and complex ANSI SQL.
• Stakeholder & Delivery Management: Proven capability managing multiple external client engagements, steering committee presentations, and senior client expectations.

Preferred Certifications

• Databricks: Databricks Certified Solutions Architect Professional / Databricks Certified Data Engineer Professional
• Microsoft Azure: Microsoft Certified: Azure Solutions Architect Expert / Azure Data Engineer Associate (DP-203)