Senior Data Engineer

Skills Group; Auckland; Hybrid; NZD 150,000 - 190,000; Full time

Benefits offered by this job

Health insurance

Hybrid work

KiwiSaver contribution

Referral bonus

Learning & development

Job summary

Skills Group is seeking a Senior Data Engineer to lead the evolution of our Azure Databricks lakehouse, expanding Delta Lake and Unity Catalog capabilities. You’ll guide architecture, data governance, and product delivery, while mentoring a small team in a hybrid Auckland-based role.

You’ll own design decisions, partner with analytics and BI, and raise engineering standards across our data platform. Strong Azure and Python skills are essential for success.

Qualifications

• Bachelor's degree or equivalent industry experience.
• Proven production experience with Azure Databricks, Delta Lake and Unity Catalog.
• Strong SQL and Python, with data modelling and scalable pipelines.
• Experience with CI/CD and infrastructure-as-code.
• Leadership or mentoring experience in agile delivery.
• Excellent stakeholder management and communication.

Responsibilities

• Lead development and evolution of Azure Databricks lakehouse architecture.
• Design and build scalable ETL/ELT pipelines in SQL and Python.
• Mature CI/CD, infrastructure-as-code, DevOps, monitoring, and observability.

Skills

Azure Databricks

Delta Lake

Python

SQL

CI/CD

Mentoring

Education

Bachelor's degree or equivalent

Tools

Azure Data Factory

Azure Data Lake Storage Gen2

Azure Key Vault

Power BI

Job description

Lead the evolution of our Azure Databricks platform and help shape the future of data engineering at Skills Group.

Skills Group is looking for an experienced Senior Data Engineer - Azure Databricks to lead the continued design, build, and delivery of our data platform. With the core foundations already in place, you'll take ownership of extending, hardening, and scaling the platform - introducing new capability, strengthening governance, and improving engineering practices as our data environment matures.

This is a senior, hands-on technical leadership role where you'll lead and grow a small team of data engineers, set technical direction and engineering standards, and partner closely with analytics, BI, and business stakeholders. Proven production experience with Azure Databricks is essential for this role.

Based at our Ellerslie office, you'll enjoy a hybrid working model with three days in the office and two days working from home each week.

What You'll Be Doing

• Lead the continued development and evolution of our Azure Databricks lakehouse architecture, including Delta Lake and Unity Catalog.
• Design and build scalable ETL/ELT pipelines using SQL and Python, and extend capability across the Azure data ecosystem.
• Mature CI/CD, infrastructure-as-code, DevOps, monitoring, and observability practices.
• Strengthen data governance, quality, access control, security, performance, and reliability.
• Lead, coach, and mentor data engineers while setting technical direction and engineering standards.
• Partner with analytics, BI, and business stakeholders to deliver trusted data solutions and contribute to the broader data strategy.

What You'll Bring

• Proven experience delivering and supporting production data platforms on Azure Databricks, including Delta Lake and Unity Catalog - this is essential.
• Strong Azure data ecosystem experience, including Azure Data Factory, Azure Data Lake Storage Gen2, and Azure Key Vault.
• Strong SQL and Python skills, with experience in data modelling and building scalable data pipelines.
• Experience with CI/CD, infrastructure-as-code, data governance, data quality, and access control.
• Experience leading or mentoring data engineers, setting technical direction, and working within agile delivery environments.
• Strong stakeholder management and communication skills, plus a relevant Bachelor's degree or equivalent industry experience.
• Nice to have: Power BI experience, streaming or near-real-time data ingestion experience, and relevant Databricks or Microsoft Azure certifications.

At Skills Group, our purpose is to maximise human potential. For more than 30 years, we've partnered with learners, employers and industry to build a skilled workforce across New Zealand. You'll join a collaborative environment where you'll have the opportunity to influence technical direction, develop your team, and play a key role in the continued evolution of our data capability.

Benefits That Back You

• Health & wellbeing: Fully subsidised Southern Cross medical insurance, TELUS Health Employee Assistance Programme (EAP), and annual flu vaccinations.
• Flexible work & leave: Hybrid working with three days in the office and two days from home, birthday leave, long service leave, and enhanced bereavement leave.
• Financial benefits: 3.5% KiwiSaver contribution, 2% SuperLife contribution, and a $500 employee referral bonus.
• Learning & development: Two paid learning hours each week, LinkedIn Learning access, Future Fit development opportunities, and ongoing professional growth support.

If you're an experienced Azure Databricks engineer ready to take technical ownership, lead others, and help mature a growing data platform, we'd love to hear from you.