Staff Data Engineer – Payments Platform (AWS/Snowflake)

Commonwealth Bank; Melbourne; On-site; AUD 180,000 - 240,000; Full time

Job summary

Commonwealth Bank in Melbourne beacons a Staff Data Engineer role within Payments Technology steering scalable data platforms and trusted data products. You’ll lead data pipelines on AWS, Snowflake, Python and Spark, shaping how Payments data is engineered, operated and made accessible across the bank.

You’ll collaborate with engineers, product teams and data consumers to deliver cloud-native, reliable data capabilities, with emphasis on batch and streaming processing, data quality and

Qualifications

• Experience designing data platforms and products at scale.
• Proficient in Spark, Airflow and cloud data services.
• Ability to influence architecture across multiple squads.
• Strong Python and PySpark coding skills.

Responsibilities

• Set technical direction for complex data engineering initiatives and influence architecture across squads.
• Design, build and evolve scalable batch and streaming data pipelines using Spark, Airflow and AWS Glue.
• Build cloud-native data solutions across AWS services including S3, EMR, Glue, Lambda, Redshift and SageMaker.
• Engineer high-quality data products and evolve enterprise data platform patterns.
• Lead engineering practices across testing, CI/CD, infrastructure-as-code, observability and production reliability.
• Establish patterns for data quality, resilience, performance and maintainability across workloads.

Skills

AWS data services

Snowflake

Python PySpark

Batch streaming pipelines

CI/CD IaC

Observability

Advanced SQL

Technical leadership

Tools

Spark

Airflow

AWS Glue

SageMaker

Redshift

S3

Job description

Commonwealth Bank in Melbourne beacons a Staff Data Engineer role within Payments Technology steering scalable data platforms and trusted data products. You’ll lead data pipelines on AWS, Snowflake, Python and Spark, shaping how Payments data is engineered, operated and made accessible across the bank.

You’ll collaborate with engineers, product teams and data consumers to deliver cloud-native, reliable data capabilities, with emphasis on batch and streaming processing, data quality and