Senior Data Engineer — Databricks & Azure Pipelines

Younity; Auckland; Presencial; NZD 120 000 - 180 000; Tempo integral

Resumo da oferta

Jora New Zealand is seeking an experienced Senior Data Engineer to design, build and optimise a modern cloud-based data platform with a strong focus on Databricks, Azure and scalable data engineering. This hands-on senior role suits someone who enjoys solving complex data problems, building robust pipelines and working across both batch and real-time data environments.

The role involves designing scalable data pipelines, developing data workflows using Python, PySpark and SQL, and working with

Qualificações

• 5+ years' experience in Data Engineering.
• At least 2 years' strong hands-on Databricks experience.
• Strong Python, PySpark and SQL capability.
• Solid experience with Azure Data Factory.
• Strong understanding of Apache Spark and Delta Lake.
• Experience with Azure Data Lake and broader Azure data services.
• Experience building production-grade batch and/or streaming pipelines.
• Good understanding of data modelling, data warehousing and lakehouse concepts.
• Experience with Git, CI/CD and modern engineering practices.

Responsabilidades

• Designing and building scalable data pipelines using Databricks and Azure Data Factory.
• Developing data workflows using Python, PySpark and SQL.
• Building and optimising ETL/ELT pipelines across cloud and on-premise data sources.
• Working with Delta Lake, CDC and lakehouse architecture.
• Developing batch and real-time data processing solutions.
• Working with Spark Structured Streaming, Event Hubs and Kafka.
• Optimising Spark workloads, Databricks jobs and data storage patterns.
• Designing data models for analytical and operational use cases.
• Improving data quality, monitoring, governance and reliability.
• Supporting CI/CD, code reviews and engineering best practices.
• Mentoring other engineers and contributing to technical standards.

Conhecimentos

Databricks

Azure Data Factory

Python

PySpark

SQL

Delta Lake

Spark

Event Hubs

Kafka

CI/CD

Git

Docker

Azure Data Lake

Lakehouse

MLOps

Data governance

Unity Catalog

IaC

Ferramentas

Docker

Descrição da oferta de emprego

Jora New Zealand is seeking an experienced Senior Data Engineer to design, build and optimise a modern cloud-based data platform with a strong focus on Databricks, Azure and scalable data engineering. This hands-on senior role suits someone who enjoys solving complex data problems, building robust pipelines and working across both batch and real-time data environments.

The role involves designing scalable data pipelines, developing data workflows using Python, PySpark and SQL, and working with