Lead Databricks Data Engineer - Lakehouse & AI Pipelines

Theta (NZ); Auckland; On-site; NZD 150,000 - 190,000; Full time

Benefits offered by this job

Hybrid working model

Fully subsidised medical insurance

Parental leave benefits

Staff Assistance Programme

Remote working options where practical

Flexible working hours

Job summary

Theta NZ is seeking an experienced Lead Data Consultant to join the Data and Insights Practice in Auckland. The role focuses on end-to-end data solution delivery with Databricks, Azure, and Microsoft Fabric, delivering reliable data pipelines and scalable platforms.

You will lead data engineering efforts, mentor engineers, and collaborate with stakeholders to shape designs, estimates, and delivery plans, ensuring high-quality outcomes for enterprise clients.

Qualifications

• Eight or more years in data engineering for enterprise-scale platforms and EDWs.
• Three or more years hands-on Databricks experience.
• Strong Databricks Workspace, Unity Catalog and Compute design and management experience.
• Experience with Databricks notebooks, workflows, Delta Live Tables and Lakeflow.
• Advanced SQL capability.
• Experience delivering modern Lakehouse, Warehouse or cloud-native data platforms.
• Understanding Delta Lake, Medallion Architecture and scalable solution design.
• Data modeling experience for analytics workloads.
• Git, CI/CD, DevOps and DataOps experience.
• Stakeholder engagement and communication skills.
• Mentoring or technical leadership.
• Experience using AI coding assistants (GitHub Copilot or Claude).
• Exposure to Microsoft Fabric and Azure Data Platform.

Responsibilities

• Design, build, and optimise data pipelines across Lakehouse and warehouse patterns, with a focus on reliability and performance.
• Develop robust data transformations that enable reporting, analytics, and AI use cases.
• Deliver production grade solutions using Databricks notebooks, workflows, Delta Live Tables and Lakeflow.
• Produce clean, maintainable SQL/Python data solution that meets agreed engineering standards.
• Implement automated testing, monitoring, data validation, and performance tuning to support stable operations.
• Apply modern data modelling and Medallion Architecture principles to create scalable, well-structured data products.
• Lead technical discovery and solution walkthroughs, translating business requirements into clear engineering scope and deliverables.
• Partner with architects and delivery teams to shape solution design, estimates, sequencing, and delivery plans.
• Provide clear communication on progress, risks, dependencies, and technical blockers to client stakeholders and delivery leadership.
• Conduct peer reviews and promote consistent patterns across coding, documentation, and deployment practices.
• Mentor team members, supporting capability uplift and delivery confidence.
• Contribute to Theta delivery assets including templates, reusable components, documentation, and continuous improvement initiatives.

Skills

Data engineering

Databricks

SQL

Python

Medallion Architecture

Data modeling

CI/CD

DevOps

DataOps

Stakeholder engagement

Leadership

Mentoring

AI coding assistants

Tools

Databricks

Unity Catalog

Delta Live Tables

Lakeflow

Microsoft Fabric

Azure

Job description

Theta NZ is seeking an experienced Lead Data Consultant to join the Data and Insights Practice in Auckland. The role focuses on end-to-end data solution delivery with Databricks, Azure, and Microsoft Fabric, delivering reliable data pipelines and scalable platforms.

You will lead data engineering efforts, mentor engineers, and collaborate with stakeholders to shape designs, estimates, and delivery plans, ensuring high-quality outcomes for enterprise clients.