AI-Driven Data Engineer — Azure Data Platform & Pipelines

Westpac Group; Sydney; On-site; AUD 140,000 - 190,000; Full time

Benefits offered by this job

Banking product discounts

Flexible work arrangements

Learning and development opportunities

Community volunteering

Job summary

Westpac Group is seeking a skilled Data Engineer to join Enterprise Platforms Engineering (EPE) and help shape how enterprise data is engineered, governed, and activated across customer, risk, and operational domains.

You will work with Hadoop, Python, PySpark, ETL, and Azure cloud technologies, delivering scalable data products and AI-enabled data platforms. Collaboration with product owners and security teams is key to success.

Qualifications

• Hands-on data engineering with cloud platforms and Azure stack.
• Strong experience in building scalable data pipelines and data products.
• Expert-level understanding of data modelling and warehousing concepts.
• Experience delivering AI/GenAI solutions in enterprise environments.

Responsibilities

• Design, build, and support cloud native, event driven data platforms and APIs.
• Develop and operate scalable data pipelines and storage solutions on Azure.
• Write high quality, secure, and maintainable code with strong performance and reliability.
• Optimize data pipelines, queries, and algorithms for scalability and cost efficiency.
• Produce and review technical designs, documentation, and runbooks.
• Engineer AI/GenAI ready datasets with data quality, lineage, and governance.
• Apply responsible AI, privacy, and data governance controls in regulated environments.
• Collaborate with delivery, product, architecture, security, and platform teams.
• Participate in code reviews, uplift engineering standards, and mentor others.
• Contribute to the strategic evolution of the data platform and engineering practices.

Skills

Hadoop

Python

PySpark

ETL

Azure cloud stack

Kafka

Java

Scala

Spring Boot

Azure Synapse

Cosmos DB

Git

Jenkins

CI/CD

Serverless architectures

Azure Data Lake

Azure Event Hubs

Snowflake

Teradata

Jira

Confluence

Data modelling

Data warehousing

GenAI

AI engineering

Prompting

RAG

Agents/tools

Tools

Jira

Confluence

Git

Job description

Westpac Group is seeking a skilled Data Engineer to join Enterprise Platforms Engineering (EPE) and help shape how enterprise data is engineered, governed, and activated across customer, risk, and operational domains.

You will work with Hadoop, Python, PySpark, ETL, and Azure cloud technologies, delivering scalable data products and AI-enabled data platforms. Collaboration with product owners and security teams is key to success.