Azure Data Lake Architect & Databricks Lead

ECI Group; Houston, Northern (TX, KY); Hybrid; USD 150,000 - 190,000; Full time

Job summary

ECI Group's Professional Technical Services Network is seeking a Principal Data Engineer in Houston, TX for a ~7-month contract starting Oct 5, 2026. You will lead Azure-based data lake solutions, governance, and high-volume analytics initiatives to drive organizational success.

The role requires 15–17 years in IT, expertise in Databricks, Delta Live Tables, Unity Catalog, and strong Python/SQL skills. Certification is preferred; collaboration with cross-functional teams is essential.

Qualifications

• Bachelor’s or Master’s degree in IT or related discipline.
• 15-17 years of IT experience delivering cost-effective cloud-based data analytics solutions.
• Proven track record in building data lake architectures and optimizing performance in big data environments.
• Azure Data Engineering or Databricks certification preferred.

Responsibilities

• Architect, design, and implement scalable Azure-based data lake solutions.
• Establish roadmaps for capability-building across technologies like Qlik Replicate, Databricks Delta Live Tables, and Unity Catalog.
• Provide technical leadership, ensuring adherence to best practices and quality standards.
• Lead development of data transformation pipelines using Databricks Delta Live Tables and Structured Streaming.
• Optimize Databricks resources for high-volume data processing within budget.
• Oversee Databricks Unity Catalog for governance, lineage, and metadata management.
• Develop and enforce robust data governance policies ensuring compliance, security, and data quality.
• Monitor data pipeline performance and resolve production issues.
• Collaborate with cross-functional teams to align project objectives with business goals.
• Translate technical challenges into actionable insights for executives.
• Mentor and develop a high-performing team skilled in Azure, Databricks, and data streaming technologies.
• Facilitate knowledge transfer sessions to empower internal teams with cutting-edge methodologies.

Skills

Azure

Databricks

Delta Live Tables

Synapse Analytics

Python

SQL

Scala

ETL pipelines

Data governance

Qlik Replicate

Qlik Sense

Education

Bachelor’s or Master’s degree in IT

Tools

Azure Data Factory

ADLS

Databricks

PowerBI

Qlik Sense

Job description

ECI Group's Professional Technical Services Network is seeking a Principal Data Engineer in Houston, TX for a ~7-month contract starting Oct 5, 2026. You will lead Azure-based data lake solutions, governance, and high-volume analytics initiatives to drive organizational success.

The role requires 15–17 years in IT, expertise in Databricks, Delta Live Tables, Unity Catalog, and strong Python/SQL skills. Certification is preferred; collaboration with cross-functional teams is essential.