Databricks Data Engineer (Data & AI)

Deloittenz; Wellington; On-site; NZD 120,000 - 180,000; Full time

Job summary

Deloitte New Zealand is seeking a Databricks Data Engineer to design, build, and maintain scalable data and AI solutions using Databricks. You will work across the engineering lifecycle, from ingestion and transformation to modelling, orchestration, deployment, and production support, leveraging Lakebase, Unity Catalog, and Databricks Apps.

Your AI-enabled mindset will drive use of Genie Code, GitHub Copilot, and other tools to accelerate delivery while upholding security and quality standards.

Qualifications

• Strong hands-on Databricks experience and Lakehouse architecture
• Proficient in Python for data engineering workflows
• Experience with CI/CD and modern software engineering practices
• Git-based collaboration and version control are essential
• Familiarity with Azure and/or AWS cloud services

Responsibilities

• Design, develop, and maintain scalable data and AI solutions on Databricks.
• Build robust data ingestion, ELT/ETL, and data processing pipelines.
• Develop transformations with Lakeflow Declarative Pipelines.
• Design Lakehouse architectures and data models for analytics and ML.
• Implement automated CI/CD pipelines and DevOps practices for Databricks deployments.
• Collaborate with architects, data scientists, and stakeholders to deliver practical solutions.

Skills

Databricks expertise

Python

CI/CD practices

Git workflow

Cloud platforms (Azure/AWS)

Tools

Databricks

Lakeflow

Azure Data Factory

Airflow

Terraform

Job description

As a Databricks Data Engineer in Deloitte , you will design, build, and maintain scalable data and AI solutions using Databricks. You will have the opportunity to work across the full engineering lifecycle, from data ingestion and transformation through to modelling, orchestration, deployment, optimisation, and production support as well as leverage the wider capabilities that Databricks offers with Lakebase, Unity Catalog, Databricks Apps etc.

Your AI-enabled engineering mindset will help you leverage Genie Code, GitHub Copilot, and other AI development tools to accelerate delivery while maintaining strong engineering, security, and quality standards.

Key Responsibilities

• Design, develop, and maintain scalable data and AI solutions on Databricks .
• Build robust data ingestion, ELT/ETL, and data processing pipelines using Databricks.
• Develop high-quality and reusable transformations through Lakeflow Declarative Pipelines.
• Design and implement Lakehouse architectures.
• Build dimensional, curated, and feature-ready data models to support reporting, analytics, machine learning, and Generative AI use cases.
• Develop and maintain integrations between Databricks and cloud platforms, APIs, databases, files, streaming platforms, and enterprise applications.
• Implement automated CI/CD pipelines and modern DevOps practices for Databricks deployments.
• Develop and orchestrate workloads using Lakeflow Jobs , Azure Data Factory, Airflow, or similar technologies.
• Implement appropriate security, governance, data quality, lineage, and monitoring controls .
• Optimise Databricks workloads for performance, scalability, reliability, and cost efficiency .
• Troubleshoot complex platform, pipeline issues across development and production environments.
• Collaborate with architects, analysts, data scientists, ML engineers, and business stakeholders to translate requirements into practical technical solutions.
• Contribute to engineering standards, reusable frameworks, technical documentation, and continuous improvement initiatives.
• Proactively leverage Genie Code, GitHub Copilot, and other AI-powered development tools across solution design, coding, testing, troubleshooting, code review, and documentation.
• Identify opportunities to use AI and automation to reduce repetitive engineering effort and accelerate the development lifecycle.

Required Experience

• Strong hands-on experience with Databricks .
• Experience designing and building modern Lakehouse architectures .
• Strong understanding of data engineering, data modelling, data warehousing, and distributed processing concepts.
• Experience building batch and/or streaming data pipelines.
• Experience with orchestration technologies such as Lakeflow Jobs , Azure Data Factory, Airflow, or similar tools .
• Understanding of cloud data services across Azure and/or AWS .
• Experience with Git, CI/CD, and modern software engineering practices .
• Experience using Python in data engineering solutions are advantageous.
• Familiarity with infrastructure-as-code technologies such as Terraform or similar.
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and stakeholder engagement skills.

Preferred Experience

• Experience with Generative AI, Machine Learning, or MLOps on Databricks.
• Experience with Databricks Declarative Automation Bundles
• Databricks certifications such as: Databricks Certified Data Engineer Associate
• Databricks Certified Data Engineer Professional

AI-Assisted Engineering

An AI-enabled engineering mindset , proactively using tools such as Genie Code, Claude, GitHub Copilot, and other AI capabilities to accelerate development, automate repetitive tasks, improve code quality, and deliver solutions more efficiently.

Good judgement when using AI-generated outputs, with a strong focus on security, maintainability, accuracy, and engineering quality .

What Success Looks Like

You are someone who can take ownership of a data engineering problem from design through to production .

You care about building solutions that are not only functional, but also scalable, maintainable, secure, automated, and well engineered .

You are comfortable working across the broader cloud, data, analytics, and AI ecosystem and are always looking for better ways of working through reusable patterns, automation, and effective use of AI-assisted development tools.

Most importantly, you combine strong technical capability with a delivery-focused and continuous-improvement mindset .

Why Join Us?

You will have the opportunity to:

• Work on exciting project for some of New Zealand's best companies.
• Work on modern, enterprise-scale Databricks Lakehouse platforms .
• Deliver solutions across Data Engineering, Analytics, AI, Machine Learning, and Generative AI .
• Solve complex, real-world data engineering challenges.
• Develop experience with modern engineering and AI-assisted development practices .
• Work alongside experienced data engineers, architects, platform specialists, and technology leaders.

Contribute