Senior Data Engineer: Test Data & Cloud Pipelines

NAB; City of Melbourne; On-site; AUD 140,000 - 190,000; Full time

Job summary

NAB is seeking a Senior Data Engineer to shape secure, automated data solutions in the Core Engineering Team. You’ll develop scalable Python services, build data pipelines using PySpark, and deploy on AWS and Azure to ensure reliability and performance.

You’ll work with stakeholders to design effective technology solutions, maintain ETL processes, and support CI/CD practices while documenting deployment guides and runbooks.

Qualifications

• Hands-on Python development in daily delivery.
• Solid understanding of cloud environments (AWS/Azure).
• Designing and optimising SQL for data manipulation and analysis.
• Data pipelines, ETL processes and distributed computing.
• CI/CD tooling (Jenkins or similar) and shell scripting.
• PySpark and EMR experience is a plus.
• Data masking or data privacy tooling is highly desirable.

Responsibilities

• Develop, enhance and maintain scalable applications, automation solutions and operational tooling using Python, following coding standards, best practices and reusable design principles.
• Build, deploy and support cloud-based applications and services across AWS and Azure for scalability, security, reliability and performance.
• Develop and maintain ETL processes, data pipelines and data integration using PySpark and EMR where applicable.
• Support CI/CD practices and software delivery processes for reliable releases.
• Create and maintain technical documentation, deployment guides and runbooks.
• Partner with stakeholders to gather requirements and design effective technology solutions.
• Proactively manage risk and ensure compliance; adhere to processes and procedures.

Skills

Python

PySpark

SQL

Cloud AWS/Azure

Data pipelines

CI/CD tooling

Shell scripting

EMR

Data masking

Communication

Tools

Jenkins

Job description

NAB is seeking a Senior Data Engineer to shape secure, automated data solutions in the Core Engineering Team. You’ll develop scalable Python services, build data pipelines using PySpark, and deploy on AWS and Azure to ensure reliability and performance.

You’ll work with stakeholders to design effective technology solutions, maintain ETL processes, and support CI/CD practices while documenting deployment guides and runbooks.