AI-Driven Databricks Data Engineer

Deloitte NZ; Wellington; Presencial; NZD 120.000 - 180.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Flexible work

Qualifications subsidies

Health insurance

Income protection

22 days leave

Social events

Well-being focus

Sustainability

Descripción de la vacante

Join Deloitte NZ as a Databricks Data Engineer to design, build, and maintain scalable data and AI solutions on Databricks. You'll work across ingestion, ELT/ETL, modelling, orchestration, deployment, and production support, leveraging Lakebase, Unity Catalog, and Databricks Apps.

Your AI-enabled engineering mindset will help you drive GenAI-powered development tools to accelerate delivery while upholding security and quality standards.

Formación

• Strong hands-on experience with Databricks.
• Experience designing and building Lakehouse architectures.
• Strong understanding of data engineering, data modelling, data warehousing, and distributed processing concepts.
• Experience building batch and/or streaming data pipelines.
• Experience with orchestration technologies such as Lakeflow Jobs, Azure Data Factory, Airflow, or similar tools.
• Understanding of cloud data services across Azure and/or AWS.
• Experience with Git, CI/CD, and modern software engineering practices.
• Experience using Python in data engineering solutions are advantageous.
• Familiarity with infrastructure-as-code technologies such as Terraform or similar.
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and stakeholder engagement skills.

Responsabilidades

• Design, develop, and maintain scalable data and AI solutions on Databricks.
• Build robust data ingestion, ELT/ETL, and data processing pipelines using Databricks.
• Develop high-quality and reusable transformations through Lakeflow Declarative Pipelines.
• Design and implement Lakehouse architectures.
• Build dimensional, curated, and feature-ready data models to support reporting, analytics, machine learning, and Generative AI use cases.
• Develop and maintain integrations between Databricks and cloud platforms, APIs, databases, files, streaming platforms, and enterprise applications.
• Implement automated CI/CD pipelines and modern DevOps practices for Databricks deployments.
• Develop and orchestrate workloads using Lakeflow Jobs, Azure Data Factory, Airflow, or similar technologies.
• Implement appropriate security, governance, data quality, lineage, and monitoring controls.
• Optimise Databricks workloads for performance, scalability, reliability, and cost efficiency.
• Troubleshoot complex platform, pipeline issues across development and production environments.
• Collaborate with architects, analysts, data scientists, ML engineers, and business stakeholders to translate requirements into practical technical solutions.
• Contribute to engineering standards, reusable frameworks, technical documentation, and continuous improvement initiatives.
• Proactively leverage Genie Code, GitHub Copilot, and other AI-powered development tools across solution design, coding, testing, troubleshooting, code review, and documentation.
• Identify opportunities to use AI and automation to reduce repetitive engineering effort and accelerate the development lifecycle.

Conocimientos

Databricks

Lakehouse architectures

Data engineering

Pipelines (ETL/ELT)

Orchestration (Lakeflow/ADF/Airflow)

Cloud platforms (Azure/AWS)

Git CI/CD

Python

IaC (Terraform)

Analytics & problem-solving

Stakeholder communication

Herramientas

Lakeflow

Databricks Apps

Descripción del empleo

Join Deloitte NZ as a Databricks Data Engineer to design, build, and maintain scalable data and AI solutions on Databricks. You'll work across ingestion, ELT/ETL, modelling, orchestration, deployment, and production support, leveraging Lakebase, Unity Catalog, and Databricks Apps.

Your AI-enabled engineering mindset will help you drive GenAI-powered development tools to accelerate delivery while upholding security and quality standards.