Senior Data Engineer

Westpac; New Zealand; On-site; NZD 120,000 - 170,000; Full time

Benefits offered by this job

4 weeks holiday + 5 wellbeing days

Banking benefits, insurance discounts

Growth and development opportunities

Recognition for performance

School holiday subsidy

Volunteer days

Job summary

Westpac seeks a Senior Data Engineer to lead and mentor in the CDC data pipelines that move data from multiple sources to Kafka and HDFS. You will partner with business stakeholders to ensure compliant, reliable data products and platforms, while aligning with risk controls.

You will translate requirements into scalable designs, drive incident response, and uplift squad engineering quality with strong DevOps practices and a focus on security.

Qualifications

• Strong programming skills in Java and Python.
• Deep experience with the Kafka stack, building real-time publishers and consumers.
• Good at the Hadoop stack, particularly HDFS and Spark.
• Experience orchestrating data workflows with Airflow.
• 5+ years of experience in software development, data engineering, platform engineering, or a related technical discipline.
• Solid understanding of CDC patterns and data streaming patterns.
• Awareness of secure software development practices for handling sensitive data
• Experience with Terraform, CI/CD pipelines and containerisation.
• Familiarity with code quality and security scanning tooling such as SonarQube and Snyk.
• Demonstrated ability to assess new technologies, review vendor solutions, and recommend improvements to existing engineering approaches.
• Proven capability in diagnosing and resolving complex technical issues throughout the application lifecycle.
• Strong understanding of quality assurance, testing frameworks, and continuous improvement.
• Tertiary qualifications in Computer Science, Information Systems, Engineering, or related discipline, with relevant technical certifications preferred.
• Demonstrated ability to mentor, support and grow engineers.
• Comfortable in Agile delivery and modern DevOps practices.

Responsibilities

• Provide technical leadership and guidance within the squad, shaping engineering practices, solution design, and platform stability.
• Ensure CDC platforms, data products, and pipelines remain reliable, secure, compliant, and aligned with organisational risk and regulatory requirements.
• Drive the resolution of complex production issues, including coordinating on-call coverage, early life support and critical incident response.
• Translate business and architecture requirements into scalable technical designs, ensuring consistency with wider Technology patterns and standards.
• Maintain and improve service operations by ensuring SLAs, monitoring, and system health metrics are consistently met.
• Contribute to squad level standards for engineering quality, tooling, coding practices, documentation, and deployment processes.
• Mentor engineers, fostering technical growth, capability uplift, and a culture of excellence across the squad.
• Support prioritisation alongside Leads, Service Owners, and Delivery Leads, advising on scope, timelines, and technical dependencies.
• Lead problem investigations, root-cause analysis, and continuous improvement initiatives to enhance platform stability and performance.
• Manage relationships with internal stakeholders, third-party vendors, and platform partners to ensure the delivery of high-quality outcomes.

Skills

Java

Python

Kafka

Hadoop stack

HDFS

Spark

Airflow

Terraform

CI/CD pipelines

containerisation

SonarQube

Snyk

Agile

DevOps practices

mentoring

vendor solution reviews

problem solving

QA/testing awareness

communication and collaboration

Education

Tertiary qualifications in Computer Science, Information Systems, Engineering, or related discipline

Tools

Kafka stack

Job description

Job Description - Senior Data Engineer

The Senior Data Engineer not only provides expert domain knowledge and expertise in the area they are working in, but as a senior member of the squad, provides mentorship and guidance to others. They operate with greater scope within the service group, wider technology business unit and with business stakeholders to ensure high levels of Service Delivery are maintained and an overarching focus on risk and compliance is always front of mind in the squad.

Communication, negotiation, collaboration and teamwork, and partnership/relationship building skills are key to achieving desired outcomes in this role. In particular to enable the delivery of great innovative banking products that continue to enhance our customers' experience.

The CDC Squad develops and maintains the data pipelines that capture changes from hundreds of tables across DB2 z/OS, DB2 LUW, VSAM, and Oracle and stream them to Kafka and HDFS. Our data supports analytics, regulatory reporting, and other engineering squads across the bank. We also do other work from time to time to support the wider service group.

What you’ll do:

• Provide technical leadership and guidance within the squad, shaping engineering practices, solution design, and platform stability.
• Ensure CDC platforms, data products, and pipelines remain reliable, secure, compliant, and aligned with organisational risk and regulatory requirements.
• Drive the resolution of complex production issues, including coordinating on-call coverage, early life support and critical incident response.
• Translate business and architecture requirements into scalable technical designs, ensuring consistency with wider Technology patterns and standards.
• Maintain and improve service operations by ensuring SLAs, monitoring, and system health metrics are consistently met.
• Contribute to squad level standards for engineering quality, tooling, coding practices, documentation, and deployment processes.
• Mentor engineers, fostering technical growth, capability uplift, and a culture of excellence across the squad.
• Support prioritisation alongside Leads, Service Owners, and Delivery Leads, advising on scope, timelines, and technical dependencies.
• Lead problem investigations, root-cause analysis, and continuous improvement initiatives to enhance platform stability and performance.
• Manage relationships with internal stakeholders, third-party vendors, and platform partners to ensure the delivery of high-quality outcomes.

Nā tāu rourou | What will you bring?

• Strong programming skills in Java and Python.
• Deep experience with the Kafkastack, building real-time publishers and consumers.
• Good at the Hadoop stack, particularly HDFS and Spark.
• Experience orchestrating data workflows with Airflow.
• 5+ years of experience in software development, data engineering, platform engineering, or a related technical discipline.
• Solid understanding of CDC patterns and data streaming patterns.
• Awareness of secure software development practices for handling sensitive data
• Experience with Terraform, CI/CD pipelines and containerisation.
• Familiarity with code quality and security scanning tooling such as SonarQube and Snyk.
• Demonstrated ability to assess new technologies, review vendor solutions, and recommend improvements to existing engineering approaches.
• Proven capability in diagnosing and resolving complex technical issues throughout the application lifecycle.
• Strong understanding of quality assurance, testing frameworks, and continuous improvement.
• Tertiary qualifications in Computer Science, Information Systems, Engineering, or related discipline, with relevant technical certifications preferred.
• Demonstrated ability to mentor, support and grow engineers.
• Comfortable in Agile delivery and modern DevOps practices.

Westpac Hei Wāhi Mahi | Working at Westpac

In addition to immersing yourself in a supportive team environment, you'll receive some great job perks and benefits including:

• 4 weeks standard holiday + 5 additional days of wellbeing leave to support you to be your best
• Additional purchased leave options up to 4 weeks per year
• Banking benefits, insurance discounts and superannuation scheme
• Growth and development - we provide opportunities for development and promote internal mobility
• Recognition - we recognise and reward our star performers
• School holiday subsidy - help you balance work and family during school holiday
• 2 Volunteer days per year

Ko mātou tēnei | Our Organisation

At Westpac, we're working together to build a fairer and stronger New Zealand for our people, customers, and communities. For over 160 years we've been driven by our purpose to Create Better Futures Together.

What makes us unique are the incredible people we're lucky enough to have walk through our doors every day. The giant 'W' on our building doesn't just stand for Westpac - it says, double you. We're a place that values, rewards and recognises individuality. Where you can be your whole self, your true self, and your best self. We're recognised as a place where talent is developed, where passions are fuelled and where ambitions of all shapes and sizes thrive. And it all starts with you!

The people of Aotearoa, New Zealand hail from many cultures and backgrounds. At Westpac New Zealand, we draw on this enormous talent pool to cultivate a team rich in ability and empowered to reach its full potential. Our aim is to be brave challengers, celebrating our differences and relentlessly advocating for our customers. It starts with us - we believe that when we care deeply and do our best for our people, they'll do their best for our customers. We encourage people from all walks of life to apply, including those who are neurodivergent, have chronic health conditions or disabilities - your attitude and passion are as important to us as your experience, qualifications and capabilities.