Databricks Lead Data Engineer – Lakehouse & AI (Auckland)

Theta; Wellington; Hybrid; NZD 150,000 - 190,000; Full time

Benefits offered by this job

Hybrid working model

Fully subsidised medical insurance for

Sick leave from first day

Volunteer days

Annual Flu injections

Remote working options

INZ accredited employer

Job summary

Theta is seeking an experienced Lead Data Consultant to join the Data and Insights Practice, based in the Auckland office. This role centers on end-to-end data solutions using Databricks, Azure and Microsoft Fabric, delivering reliable data pipelines and models.

You will collaborate with client stakeholders, solution architects, and delivery teams, mentoring juniors and contributing to reusable assets and templates to elevate delivery quality.

Qualifications

• Eight+ years delivering enterprise-scale data platforms and EDWs.
• Three+ years hands-on Databricks experience.
• Strong Databricks Workspace, Unity Catalog and Compute design.
• Experience with Databricks notebooks, workflows, Delta Live Tables and Lakeflow.
• Advanced SQL capability.
• Experience delivering modern Lakehouse/cloud data platforms.
• Knowledge of Delta Lake and Medallion Architecture.
• Git, CI/CD, DevOps and DataOps experience.
• Stakeholder engagement and communication skills.
• Mentoring and technical leadership experience.
• Experience using AI coding assistants (GitHub Copilot/Claude).
• Databricks certifications or ongoing development commitment.
• Exposure to Microsoft Fabric and Azure Data Platform.

Responsibilities

• Design, build, and optimise data pipelines with reliability and performance.
• Develop robust data transformations for reporting, analytics, and AI use cases.
• Deliver production-grade solutions using Databricks notebooks, workflows, Delta Live Tables and Lakeflow.
• Produce clean, maintainable SQL/Python data solutions meeting engineering standards.
• Implement automated testing, monitoring, data validation, and performance tuning.
• Apply Medallion Architecture for scalable data products.
• Lead technical discovery and translate business requirements into engineering scope.
• Partner with architects and delivery teams on design, estimates, sequencing, and delivery plans.
• Provide clear communication on progress, risks, and blockers to stakeholders.
• Mentor team members and contribute to reusable assets and templates.

Skills

Data engineering

Databricks

SQL

Python

Data modelling

CI/CD

Stakeholder engagement

Mentoring

Azure

Git

Tools

Databricks

Delta Live Tables

Lakeflow

Unity Catalog

GitHub Copilot

Job description

Theta is seeking an experienced Lead Data Consultant to join the Data and Insights Practice, based in the Auckland office. This role centers on end-to-end data solutions using Databricks, Azure and Microsoft Fabric, delivering reliable data pipelines and models.

You will collaborate with client stakeholders, solution architects, and delivery teams, mentoring juniors and contributing to reusable assets and templates to elevate delivery quality.