Staff Data Engineer

Commonwealth Bank; Sydney; On-site; AUD 180,000 - 240,000; Full time

Job summary

Commonwealth Bank in Australia is seeking a Staff Data Engineer for Payments Technology, based in Sydney or Melbourne. You will provide hands-on technical leadership for data platforms, data products, and pipelines that support Payments, solving complex problems at scale and influencing architecture across squads.

You will design and build batch and streaming pipelines using Spark, Airflow and AWS Glue, and develop cloud-native solutions on AWS services such as S3, EMR, Redshift and SageMaker.

Qualifications

• Proven experience building scalable data platforms and pipelines.
• Strong knowledge of AWS data services and data products.
• Experience with batch and streaming data processing.

Responsibilities

• Set technical direction for complex data engineering initiatives and influence architecture and standards across multiple squads.
• Design, build and evolve scalable batch and streaming data pipelines using Spark, Airflow and AWS Glue.
• Build cloud-native data solutions across AWS services (S3, EMR, Glue, Lambda, Redshift, SageMaker).
• Engineer high-quality data products and help evolve enterprise data platform and data marketplace patterns.
• Lead engineering practices across testing, CI/CD, infrastructure-as-code, observability and production reliability.
• Mentor engineers, support technical decision-making and raise data engineering capability across teams.
• Partner with Architecture, Product, Security and data consumers to translate complex requirements into pragmatic engineering solutions.

Skills

AWS data services

Python

PySpark

Spark

Airflow

Snowflake

SQL

CI/CD

Observability

Technical leadership

Tools

AWS Glue

S3

Redshift

EMR

SageMaker

Lambda

Job description

Staff Data Engineer – Payments Technology

Sydney or Melbourne location

• Shape scalable data platforms and engineering standards across one of CommBank’s most critical technology domains.
• Build trusted data products and pipelines with AWS, Snowflake, Python and modern distributed data technologies.
• Influence how Payments data is engineered, operated and made available for reuse across the bank.

Do Work That Matters

Payments runs and evolves one of CommBank’s most important customer and operational flows, where resilience, safety and disciplined engineering are fundamental. Within Payments and Enterprise Transformation, the focus is also on connecting platforms, data, controls and transformation so change can be delivered with better outcomes for customers.

As a Staff Data Engineer, you’ll provide hands‑on technical leadership for the data platforms, data products and pipelines that support Payments. You’ll solve complex engineering problems at scale, influence architecture and engineering standards across squads, and help create reliable, reusable data capabilities.

There is meaningful modernisation underway. Payments has already delivered a priority data product through Marketplace as part of work testing CommBank’s enterprise data model and the patterns needed to scale trusted, reusable data products.

See yourself in our team

You’ll work with engineers, architects, product teams and data consumers to design and evolve modern data capabilities for Payments. Your focus will span cloud data engineering, distributed processing, batch and streaming pipelines, data platform patterns and production reliability.

As a Staff Data Engineer, you will:

• Set technical direction for complex data engineering initiatives and influence architecture and standards across multiple squads.
• Design, build and evolve scalable batch and streaming data pipelines, using technologies such as Spark, Airflow and AWS Glue.
• Build cloud‑native data solutions across AWS, including services such as S3, EMR, Glue, Lambda, Redshift and SageMaker.
• Engineer high‑quality data products and help evolve enterprise data platform and data marketplace patterns.
• Lead engineering practices across testing, CI/CD, infrastructure‑as‑code, observability and production reliability.
• Establish patterns for data quality, resilience, performance and maintainability across distributed data workloads.
• Mentor engineers, support technical decision‑making and raise data engineering capability across teams.
• Partner with Architecture, Product, Security and data consumers to translate complex requirements into pragmatic engineering solutions.

We’re interested in hearing from people who:

You’re an experienced data engineer with deep technical capability and a track record of influencing engineering beyond your immediate team. You enjoy solving difficult data problems, setting engineering direction and building platforms that teams can operate confidently at scale.

You’ll bring strong experience across several of the following:

• AWS data services, particularly S3, EMR, Glue, Lambda, Redshift, SageMaker or comparable cloud technologies.
• Snowflake, enterprise‑scale data platforms, data products or data marketplace patterns.
• Python and PySpark, with strong experience in distributed data processing.
• Designing and operating batch and streaming pipelines using technologies such as Airflow, Spark and Glue.
• CI/CD, automated testing and infrastructure‑as‑code, with an engineering mindset covering the complete delivery lifecycle.
• Observability, monitoring and reliability engineering for production data platforms and pipelines.
• Advanced SQL and strong experience with relational database technologies.
• Technical leadership across architecture and design, including mentoring engineers and influencing engineering standards across teams.

Working with us:

At CommBank, we're committed to creating an accessible, inclusive and respectful workplace. If you require support or adjustments, please let us know. We welcome applications from people of all backgrounds and we're particularly committed to making a positive difference for Aboriginal and/or Torres Strait Islander Peoples. For support please contact 1800 989 696.

If you're already part of the Commonwealth Bank Group (including Bankwest, x15ventures), you'll need to apply through Sidekick to submit a valid application. We’re keen to support you with the next step in your career.

We're aware of some accessibility issues on this site, particularly for screen reader users. We want to make finding your dream job as easy as possible, so if you require additional support please contact HR Direct on 1800 989 696.

Advertising End Date: 06/11/2026