Lead Data Engineer

Recru; Spring (TX); On-site; USD 130,000 - 170,000; Full time

Job summary

Recru is seeking an experienced Lead Data Engineer to design, develop, and implement scalable data products on the Databricks and Azure stack. You will mentor engineering teams, drive adoption of Databricks technologies, and deliver enterprise Lakehouse solutions that support analytics, AI, and reporting.

Collaborate with product managers, analytics teams, and business stakeholders to build secure, high-quality data products that deliver measurable business value across the organization.

Qualifications

• Bachelor's degree or equivalent professional experience.
• 7+ years of experience in Data Engineering, Data Platform Engineering and Data Architecture.
• Expert-level knowledge of Databricks, Spark SQL, SQL, Python / PySpark.
• Strong expertise in dimensional modeling, data warehouse design, and semantic modeling.
• Experience implementing enterprise Lakehouse and data platform architectures.
• Hands-on experience with Unity Catalog, Auto Loader, Databricks Workflows, DBT, Lakeflow Pipelines.
• Experience implementing CI/CD pipelines and DevOps using Azure DevOps, Git, and IaC.
• Strong understanding of data quality, governance, observability, monitoring, and operational support.
• Excellent analytical, problem-solving, communication, and stakeholder management skills.
• Proven experience mentoring engineers and leading technical delivery teams.

Responsibilities

• Design, develop, and implement scalable data products and solutions.
• Mentor engineering teams and drive adoption of modern Databricks technologies.
• Collaborate with Product, Project, Analytics, and business stakeholders to build scalable data platforms.
• Deliver governed, secure, and high-quality data products that drive business value.
• Operate within enterprise architecture standards and promote best practices in data engineering.

Skills

Databricks

Spark SQL

SQL

Python / PySpark

Dimensional modeling

Data warehouse design

Data architecture

CI/CD

DevOps

Mentoring engineers

Stakeholder management

Education

Bachelor’s degree in computer science, Information Systems, Engineering, Data Analytics, or related field

Tools

Unity Catalog

Auto Loader

Databricks Workflows

DBT

Lakeflow Pipelines

Azure DevOps

Git

Job description

We are seeking an experienced and highly motivated Lead Data Engineer to design, develop, and implement scalable data products and solutions utilizing the Databricks Data Intelligence Platform and Microsoft Azure. This role is ideal for a hands‑on technical individual who combines deep expertise in data engineering, dimensional modeling, platform development, and modern data engineering practices and agentic code generation with a passion for emerging capabilities within the Databricks ecosystem.

The successful candidate will be responsible for developing reusable engineering frameworks, implementing enterprise data solutions, mentoring engineering teams, and driving adoption of modern Databricks technologies including Azure Data Factory, Lakeflow, Lakebase, Agentbricks, Genie, Genie Spaces, Genie Code, Databricks Apps, and AI Agent Frameworks. Working within established enterprise architecture standards, this individual will play a critical role in delivering scalable Lakehouse solutions that support analytics, reporting, artificial intelligence, machine learning, and agentic applications across the organization. The Lead Data Engineer will collaborate closely with Product Managers, Project Managers, Data Engineers, Analytics teams, and business stakeholders to build modern, scalable, secure, and high‑quality data products that drive business value.

Required Qualifications

• Bachelor’s degree in computer science, Information Systems, Engineering, Data Analytics, or a related field, or equivalent professional experience.
• 7+ years of experience in Data Engineering, Data Platform Engineering and Data Architecture.
• Demonstrated experience leading technical architecture and solution design initiatives.
• Expert‑level knowledge of:
  • Databricks
  • Spark SQL
  • SQL
  • Python / PySpark
• Strong expertise in dimensional modeling, data warehouse design, and semantic modeling.
• Experience implementing enterprise Lakehouse and data platform architectures.
• Hands‑on experience with:
  • Unity Catalog
  • Auto Loader
  • Databricks Workflows
  • DBT
  • Lakeflow Pipelines
• Experience implementing CI/CD pipelines and DevOps practices using Azure DevOps, Git, and Infrastructure as Code.
• Strong understanding of data quality, governance, observability, monitoring, and operational support.
• Excellent analytical, problem‑solving, communication, and stakeholder management skills.
• Proven experience mentoring engineers and leading technical delivery teams.

Preferred Qualifications

• At least 7-10+ years related experience
• Experience implementing AI-driven and agentic solutions within Databricks.
• Experience with Databricks Mosaic AI, Vector Search, Agent Bricks, Genie, and Databricks Apps.
• Experience building Retrieval-Augmented Generation (RAG) and semantic retrieval solutions.
• Knowledge of Azure Synapse Analytics and modern cloud‑native data architectures.
• Experience supporting Power BI semantic models and enterprise reporting environments.
• Azure and Databricks certifications a plus.
• Experience designing reusable engineering frameworks and platform accelerators.

Success Profile

The ideal candidate is a influential technical contributor who can effectively balance architecture, engineering, mentorship, and architecture. They possess a deep understanding of enterprise data architecture and dimensional modeling while remaining actively engaged in solution development. This individual thrives in a fast‑paced environment, drives engineering excellence, embraces innovation, and enables the organization to leverage modern Databricks, AI, and Lakehouse technologies to deliver scalable, governed, and business‑focused data products