Lead Data Engineer — Databricks Lakehouse & Azure

Compunnel, Inc.

Houston (TX)

On-site

USD 150,000 - 185,000

Full time

14 days+

Application generator

Turn this role into an interview — a resume and cover letter built around what this employer wants.

Get past ATS filters

Job summary

Compunnel, Inc. seeks a Lead Data Engineer to provide hands-on technical leadership for an enterprise data migration to Oracle Fusion Cloud.

You will own solution architecture, delivery planning, data modeling, and Databricks Lakehouse development while guiding a team to build scalable pipelines. The ideal candidate has deep Databricks, DBT, dimensional modeling, and Azure data engineering experience, with a track record delivering enterprise data initiatives and collaborating with BI teams to

Qualifications

12+ years of overall Data Engineering experience.
6+ years of professional experience working in the United States.
10+ years of hands-on Data Engineering experience.
5+ years of hands-on experience with Databricks.
Strong expertise with Databricks Lakehouse architecture and principles.
Hands-on experience with Unity Catalog.
Experience developing Spark Declarative Pipelines using Lakeflow/DLT and Auto Loader.
Strong pipeline development experience.
Extensive experience with DBT model development, conversion, and refactoring.
Advanced expertise in dimensional data modeling.
Experience optimizing Delta tables for analytics and reporting.
Strong SQL and PySpark development skills.
Experience with medallion architecture.
Experience with Azure DevOps CI/CD pipelines and Git.
Exposure to Power BI and analytics consumption patterns.
Experience with metadata-driven frameworks.
Exposure to code generation and agentic engineering patterns.
Experience optimizing data models for Power BI, including reducing downstream DAX complexity through upstream transformations.
Familiarity with Oracle Fusion ERP data structures, BICC extraction patterns, or complex financial data domains.
Strong analytical, problem-solving, and root cause analysis skills.
Excellent verbal and written communication skills.
Ability to collaborate effectively with cross-functional teams and technical stakeholders.
Strong ownership, accountability, and ability to work independently.
Ability to proactively identify risks, dependencies, and project blockers.

Responsibilities

Lead the technical implementation, solution architecture, and delivery of enterprise data engineering initiatives.
Provide technical leadership, mentoring, and coordination for a team of data engineers.
Develop delivery plans, estimates, and work management strategies for project execution.
Design, develop, and optimize Databricks Lakehouse solutions using medallion architecture.
Lead pipeline development, DBT model development, conversion, and refactoring efforts.
Design and maintain dimensional data models optimized for reporting and analytics.
Optimize Gold-layer Delta tables for high-performance Power BI consumption.
Develop Spark Declarative Pipelines using Lakeflow/DLT and Auto Loader.
Implement and manage Unity Catalog for secure data governance.
Collaborate with BI and QA teams to ensure data quality and successful delivery.
Participate in technical design discussions and architecture reviews.
Support migration of datasets, data products, and Power BI reports to Oracle Fusion data models.
Partner with QA teams to define testing criteria and validate data accuracy.
Communicate blockers, risks, sprint progress, and delivery timelines to stakeholders.
Identify dependencies and resolve technical challenges.
Perform root cause analysis for data issues and implement solutions.
Contribute to engineering standards, metadata-driven frameworks, and continuous improvement.