lead data engineer (databricks)

Randstad Digital New Zealand; Wellington; On-site; NZD 120,000 - 180,000; Full time

Job summary

Randstad Digital New Zealand seeks a Lead Data Engineer (Azure & Databricks) to shape the technical vision for a next-generation enterprise data platform in Wellington on a fixed-term through June 2027. You will blend hands-on engineering with architectural guidance to deliver scalable, secure cloud data services that underpin critical decision-making across the public sector.

You will drive cloud-native practices across Azure, Databricks, and containerized environments while mentoring an Agile

Qualifications

• Proven experience designing and delivering enterprise cloud data platforms at scale.
• Deep expertise in Azure Databricks, Spark, Delta Lake, and Terraform.
• Strong knowledge of cloud-native architecture, containers, and security integrations.

Responsibilities

• Lead technical design and delivery of Azure data services, Databricks, and streaming solutions.
• Champion DevOps, IaC (Terraform, Azure DevOps), CI/CD pipelines, and observability.
• Provide technical mentorship to uplift data engineering capability.
• Partner with architects and stakeholders to align platform features with goals.
• Evolve platform architecture to support secure data sharing, analytics, and governance.
• Drive governance, platform standardisation, and cloud-first delivery practices.

Skills

Azure Databricks

Delta Lake

Terraform (IaC)

Docker

AKS

CI/CD

Data Platform Architecture

Tools

Terraform

Docker

AKS

Azure DevOps

Spark

Delta Lake

Job description

• Technical Lead, Building scalable enterprise Data Platforms, Data governance/Metadata
• Azure Databricks, Terraform (IaC), Data Platform, Delta Lake, CI/CD, Docker/AKS
• Wellington , Fixed Term till June 2027 (potential extension for a multi-year program)

About The Role

We are seeking a Lead Data Engineer (Azure & Databricks) to shape the technical vision and architecture of a next-generation enterprise data platform. In this position, you will guide the design and delivery of scalable, secure, and cost-effective cloud data services that underpin critical decision-making across the public sector.

As a technical lead, you will blend hands-on engineering with strategic architectural guidance. You will drive cloud-native best practices across Azure, Databricks, and containerised environments, while mentoring an Agile team of data engineers. Partnering closely with architects and business stakeholders, you will ensure the platform evolves to meet modern data governance, analytics, and integration standards.

Key Responsibilities

• Lead technical design and delivery of Azure data services, Databricks, and streaming solutions.
• Champion DevOps, IaC (Terraform, Azure DevOps), CI/CD pipelines, and proactive observability.
• Provide technical mentorship and hands-on guidance to uplift data engineering capability.
• Partner with architects and stakeholders to align platform features with organisational goals.
• Evolve platform architecture to support secure data sharing, analytics, and governance.
• Drive technical governance, platform standardisation, and cloud-first delivery practices.

Skills & Experience

• Proven experience designing and delivering enterprise cloud data platforms at scale.
• Deep expertise in Azure Databricks, Apache Spark, Delta Lake, and Terraform (IaC).
• Strong knowledge of cloud-native architecture, containerization (AKS, Docker), and security integrations.
• Experience with modern data governance tools and compliance frameworks.
• Proven background mentoring engineers and leading technical delivery within Agile environments.
• Excellent communication skills to translate cloud data strategies to varied audiences.

Please note, we can only consider applicants that are legally entitled to work in New Zealand at the time of application.

At Randstad, we are passionate about providing equal employment opportunities and embracing diversity to the benefit of all. We actively encourage applications from any background.