Lead Data Engineer

Compunnel, Inc.; Houston (TX); On-site; USD 150,000 - 185,000; Full time

Job summary

Compunnel, Inc. seeks a Lead Data Engineer to provide hands-on technical leadership for an enterprise data migration to Oracle Fusion Cloud.

You will own solution architecture, delivery planning, data modeling, and Databricks Lakehouse development while guiding a team to build scalable pipelines. The ideal candidate has deep Databricks, DBT, dimensional modeling, and Azure data engineering experience, with a track record delivering enterprise data initiatives and collaborating with BI teams to

Qualifications

• 12+ years of overall Data Engineering experience.
• 6+ years of professional experience working in the United States.
• 10+ years of hands-on Data Engineering experience.
• 5+ years of hands-on experience with Databricks.
• Strong expertise with Databricks Lakehouse architecture and principles.
• Hands-on experience with Unity Catalog.
• Experience developing Spark Declarative Pipelines using Lakeflow/DLT and Auto Loader.
• Strong pipeline development experience.
• Extensive experience with DBT model development, conversion, and refactoring.
• Advanced expertise in dimensional data modeling.
• Experience optimizing Delta tables for analytics and reporting.
• Strong SQL and PySpark development skills.
• Experience with medallion architecture.
• Experience with Azure DevOps CI/CD pipelines and Git.
• Exposure to Power BI and analytics consumption patterns.
• Experience with metadata-driven frameworks.
• Exposure to code generation and agentic engineering patterns.
• Experience optimizing data models for Power BI, including reducing downstream DAX complexity through upstream transformations.
• Familiarity with Oracle Fusion ERP data structures, BICC extraction patterns, or complex financial data domains.
• Strong analytical, problem-solving, and root cause analysis skills.
• Excellent verbal and written communication skills.
• Ability to collaborate effectively with cross-functional teams and technical stakeholders.
• Strong ownership, accountability, and ability to work independently.
• Ability to proactively identify risks, dependencies, and project blockers.

Responsibilities

• Lead the technical implementation, solution architecture, and delivery of enterprise data engineering initiatives.
• Provide technical leadership, mentoring, and coordination for a team of data engineers.
• Develop delivery plans, estimates, and work management strategies for project execution.
• Design, develop, and optimize Databricks Lakehouse solutions using medallion architecture.
• Lead pipeline development, DBT model development, conversion, and refactoring efforts.
• Design and maintain dimensional data models optimized for reporting and analytics.
• Optimize Gold-layer Delta tables for high-performance Power BI consumption.
• Develop Spark Declarative Pipelines using Lakeflow/DLT and Auto Loader.
• Implement and manage Unity Catalog for secure data governance.
• Collaborate with BI and QA teams to ensure data quality and successful delivery.
• Participate in technical design discussions and architecture reviews.
• Support migration of datasets, data products, and Power BI reports to Oracle Fusion data models.
• Partner with QA teams to define testing criteria and validate data accuracy.
• Communicate blockers, risks, sprint progress, and delivery timelines to stakeholders.
• Identify dependencies and resolve technical challenges.
• Perform root cause analysis for data issues and implement solutions.
• Contribute to engineering standards, metadata-driven frameworks, and continuous improvement.

Skills

Databricks

DBT

Dimensional modeling

SQL

PySpark

Power BI

Lakehouse architecture

Unity Catalog

Lakeflow/DLT

Auto Loader

Azure DevOps CI/CD

Git

Oracle Fusion ERP data structures

Tools

Microsoft Azure

Power BI tooling

Job description

Job Summary

We are seeking a Lead Data Engineer to provide hands-on technical leadership for a strategic enterprise data migration initiative supporting an Oracle Fusion Cloud implementation. This role is responsible for solution architecture, delivery planning, technical leadership, data modeling, and pipeline development while leading a team of data engineers to build scalable Databricks Lakehouse solutions. The ideal candidate will have deep expertise in Databricks, DBT, dimensional modeling, financial data domains, and Azure-based data engineering technologies.

Key Responsibilities

• Lead the technical implementation, solution architecture, and delivery of enterprise data engineering initiatives.
• Provide technical leadership, mentoring, and coordination for a team of data engineers.
• Develop delivery plans, estimates, and work management strategies for project execution.
• Design, develop, and optimize Databricks Lakehouse solutions using medallion architecture.
• Lead pipeline development, DBT model development, conversion, and refactoring efforts.
• Design and maintain dimensional data models optimized for reporting and analytics.
• Optimize Gold-layer Delta tables for high-performance Power BI consumption.
• Develop Spark Declarative Pipelines using Lakeflow/DLT and Auto Loader.
• Implement and manage Unity Catalog for secure data governance.
• Collaborate with Business Intelligence and Quality Assurance teams to ensure data quality and successful solution delivery.
• Participate in technical design discussions and architecture reviews.
• Support migration of datasets, data products, and Power BI reports from legacy financial models to Oracle Fusion data models.
• Partner with QA teams to define testing criteria and validate data accuracy.
• Communicate technical blockers, project risks, sprint progress, and delivery timelines to project stakeholders.
• Identify project dependencies and proactively resolve technical challenges.
• Perform root cause analysis for data issues and implement effective solutions.
• Contribute to engineering standards, metadata-driven frameworks, and continuous improvement initiatives.

Required Qualifications

• 12+ years of overall Data Engineering experience.
• 6+ years of professional experience working in the United States.
• 10+ years of hands-on Data Engineering experience.
• 5+ years of hands-on experience with Databricks.
• Strong expertise with Databricks Lakehouse architecture and principles.
• Hands-on experience with Unity Catalog.
• Experience developing Spark Declarative Pipelines using Lakeflow/DLT and Auto Loader.
• Strong pipeline development experience.
• Extensive experience with DBT model development, conversion, and refactoring.
• Advanced expertise in dimensional data modeling.
• Experience optimizing Delta tables for analytics and reporting.
• Strong SQL and PySpark development skills.
• Experience with medallion architecture.
• Experience with Azure DevOps CI/CD pipelines and Git.
• Exposure to Power BI and analytics consumption patterns.
• Experience with metadata-driven frameworks.
• Exposure to code generation and agentic engineering patterns.
• Experience optimizing data models for Power BI, including reducing downstream DAX complexity through upstream transformations.
• Familiarity with Oracle Fusion ERP data structures, BICC extraction patterns, or complex financial data domains.
• Strong analytical, problem-solving, and root cause analysis skills.
• Excellent verbal and written communication skills.
• Ability to collaborate effectively with cross-functional teams and technical stakeholders.
• Strong ownership, accountability, and ability to work independently.
• Ability to proactively identify risks, dependencies, and project blockers.

Preferred Qualifications

• Experience leading enterprise Oracle Fusion Cloud migration initiatives.
• Experience leading teams of Data Engineers.
• Experience working with financial data, ERP systems, and enterprise reporting solutions.
• Experience supporting Business Intelligence and Quality Assurance teams throughout the software delivery lifecycle.
• Experience with Power BI semantic model optimization and refactoring.