Lead Data Engineer: Databricks Lakehouse & AI

Recru; Spring (TX); On-site; USD 130,000 - 170,000; Full time

Job summary

Recru is seeking an experienced Lead Data Engineer to design, develop, and implement scalable data products on the Databricks and Azure stack. You will mentor engineering teams, drive adoption of Databricks technologies, and deliver enterprise Lakehouse solutions that support analytics, AI, and reporting.

Collaborate with product managers, analytics teams, and business stakeholders to build secure, high-quality data products that deliver measurable business value across the organization.

Qualifications

• Bachelor's degree or equivalent professional experience.
• 7+ years of experience in Data Engineering, Data Platform Engineering and Data Architecture.
• Expert-level knowledge of Databricks, Spark SQL, SQL, Python / PySpark.
• Strong expertise in dimensional modeling, data warehouse design, and semantic modeling.
• Experience implementing enterprise Lakehouse and data platform architectures.
• Hands-on experience with Unity Catalog, Auto Loader, Databricks Workflows, DBT, Lakeflow Pipelines.
• Experience implementing CI/CD pipelines and DevOps using Azure DevOps, Git, and IaC.
• Strong understanding of data quality, governance, observability, monitoring, and operational support.
• Excellent analytical, problem-solving, communication, and stakeholder management skills.
• Proven experience mentoring engineers and leading technical delivery teams.

Responsibilities

• Design, develop, and implement scalable data products and solutions.
• Mentor engineering teams and drive adoption of modern Databricks technologies.
• Collaborate with Product, Project, Analytics, and business stakeholders to build scalable data platforms.
• Deliver governed, secure, and high-quality data products that drive business value.
• Operate within enterprise architecture standards and promote best practices in data engineering.

Skills

Databricks

Spark SQL

SQL

Python / PySpark

Dimensional modeling

Data warehouse design

Data architecture

CI/CD

DevOps

Mentoring engineers

Stakeholder management

Education

Bachelor’s degree in computer science, Information Systems, Engineering, Data Analytics, or related field

Tools

Unity Catalog

Auto Loader

Databricks Workflows

DBT

Lakeflow Pipelines

Azure DevOps

Git

Job description

Recru is seeking an experienced Lead Data Engineer to design, develop, and implement scalable data products on the Databricks and Azure stack. You will mentor engineering teams, drive adoption of Databricks technologies, and deliver enterprise Lakehouse solutions that support analytics, AI, and reporting.

Collaborate with product managers, analytics teams, and business stakeholders to build secure, high-quality data products that deliver measurable business value across the organization.