Principal Data Engineer (Aws)

Commonwealth Bank Of Australia

Western Australia

Hybrid

AUD 180,000 - 280,000

Full time

13 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Commonwealth Bank Of Australia is seeking a Principal Data Engineer (AWS) to shape the engineering direction for platforms enabling data consumption, model execution and deployment at scale. You will lead capital provisioning modelling platforms, ensure governance, and drive migrations from legacy systems across the data stack.

You will work across credit risk modelling, data and platform engineering, contributing to architecture, security, and reliability while mentoring engineers and raising

Qualifications

  • Proven experience building cloud-native data and model platforms at scale.
  • Experience productionising ML workloads and models.
  • Ability to design reusable data and model platform components.
  • Strong understanding of data lineage, controls and auditability in financial systems.

Responsibilities

  • Define and drive platform strategy for capital provisioning modelling.
  • Architect and build cloud-native platforms (AWS) for models and data.
  • Lead engineering patterns for model deployment, orchestration and monitoring.
  • Ensure alignment with regulatory and audit requirements including data lineage and reproducibility.
  • Mentor engineers and uplift engineering standards.
  • Collaborate with modelling teams to productionise models and improve reliability.

Skills

AWS
SageMaker
Python
PySpark
SQL
Data governance

Tools

Airflow
Spark
Glue
Snowflake
Terraform
CI/CD

Job description

Principal Data Engineer (AWS)
  • You are passionate about solving complex problem with the latest tech
  • We're the largest and most advanced Data Engineering teams in the country.
  • Together we build tomorrow's bank with world class engineering
Do work that matters

The Capital Provisioning Modelling Crew builds and runs the platforms that support how credit risk models are developed, validated, implemented, and monitored aligning to the Model Development Lifecycle Methodology (MDLC), processes, and procedures. The outcomes produced by these models are used in managing the Group Risk Management functions including front line business processes as well as for regulatory reporting including loss provisioning, portfolio analysis, and reporting.

These are critical, model-driven systems operating under regulatory scrutiny, where accuracy, control and reliability are essential.

As a Principal Engineer, you will shape the engineering direction for platforms that enable data consumption, model execution and deployment at scale, while meeting strong governance and control requirements.

See yourself in the team

You'll be part of the Capital Provisioning Modelling Crew, working across credit risk modelling, data and platform engineering.

The crew focuses on:
  • Consuming trusted data from enterprise data marketplace
  • Enabling model execution and orchestration at scale
  • Supporting SageMaker-based model deployment and ML workloadsDriving modernisation and decommissioning of legacy and sandpit environments
  • Delivering efficiencies into the framework and tools that can help deliver the models in a safe, sound, and secure way
  • Embedding controls, lineage and auditability by design
As a Principal Engineer, you will
  • Define and drive platform strategy for capital provisioning modelling
  • Architect and build cloud-native platforms (AWS) supporting model and data workloads
  • Provide thought leadership and contributes to the design of technical solutions and consumption patterns, including data security and data management.
  • Design solutions that consume and operationalise enterprise data marketplace assets
  • Lead engineering patterns for model and ML deployment, orchestration and monitoring
  • Drive migration from legacy platforms to strategic, governed environments
  • Ensure alignment with Model Risk Management, regulatory and audit requirements, including:
    • Data lineage and traceability
    • Reproducibility of model outputs
    • Strong controls and validation
  • Partner with modelling teams to productionise models and improve reliability and performance
  • Influence architecture and engineering decisions across crews and domains
  • Set and uplift engineering standards and practices
  • Mentor engineers and contribute to a high-performing engineering culture
We're interested in hearing from people who
  • Have strong experience building cloud-native data and model platforms at scale
  • Have experience productionising models or ML workloads
  • Have experience in building frameworks and tools that can significantly improve engineering productivity
  • Understand data integrity, reconciliation, and accuracy in financial systems
  • Can influence across teams and align stakeholders on technical direction
  • Have a natural drive to educate, communicate, and coordinate with different internal stakeholders and consultants.
  • Are comfortable balancing technical debt with long-term platform strategy
Technical experience

We use a broad range of tools, languages, and frameworks. We do not expect you to know them all but experience or exposure with some of these, or a willing to learn these skillsets (or equivalents) will set you up for success in this team!

  • AWS (S3, EMR, Glue, Lambda, Redshift, SageMaker or similar)
  • Snowflake or enterprise data platforms / data marketplace patterns
  • Python, PySpark and distributed data processing
  • Batch and streaming data pipelines (e.g. Airflow, Spark, Glue)
  • CI/CD, automated testing and infrastructure-as-code
  • Observability, monitoring and reliability practices
  • Strong SQL and relational database experience
Working with us

Our people bring their diverse backgrounds and unique perspectives to build a respectful, inclusive, and flexible workplace with flexible work locations. One where we're driven by our values, and supported to share ideas, initiatives, and energy. One where making a positive impact for customers, communities and each other is part of our every day.

Here, you'll thrive. You'll be supported when faced with challenges and empowered to tackle new opportunities. We're hiring engineers from across all of Australia and have opened technology hubs in Melbourne and Perth. We really love working here, and we think you will too.

We support our people with the flexibility to balance where work is done with at least half their time each month connecting in office. We also have many other flexible working options available including changing start and finish times, part-time arrangements and job share to name a few. Talk to us about how these arrangements might work in the role you're interested in.

We're aware of some accessibility issues on this site, particularly for screen reader users. We want to make finding your dream job as easy as possible, so if you require additional support please contact HR Direct on 1800 989 696.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Principal Data Engineer (Aws)
Principal Data Engineer (Aws)

Commonwealth Bank • Melbourne

Hybrid
AUD 180,000 - 240,000
Flexible work locations
Work-life balance
Principal Data Engineer
Principal Data Engineer

Medibank Private Limited • Australia

On-site
AUD 180,000 - 240,000
Principal Software Engineer - Institutional Lending, Pricing and ESG
Principal Software Engineer - Institutional Lending, Pricing and ESG

Commonwealth Bank • Sydney

On-site
AUD 180,000 - 280,000
Principal Engineer - Cloud & Platform Engineering
Principal Engineer - Cloud & Platform Engineering

Westpac Group • Sydney

On-site
AUD 210,000 - 270,000
Mortgage rates for employees
Flexible work arrangements
Wellbeing & culture leave
+2
Principal Engineer - Cloud & Platform Engineering
Principal Engineer - Cloud & Platform Engineering

Westpac Banking Corporation • Sydney

Hybrid
AUD 250,000 - 380,000
Mortgage discounts
Flexible work arrangements
Professional development
+1
Engineering Executive Manager-Group Security (Data And Software)
Engineering Executive Manager-Group Security (Data And Software)

Commonwealth Bank Of Australia • Australia

Hybrid
AUD 250,000 - 350,000
Principal Engineer Data and AI
Principal Engineer Data and AI

Westpac Group • Sydney

Hybrid
AUD 180,000 - 260,000
Special offers on banking products
Flexible work arrangements
Learning and development opportunities
+1
Engineering Executive Manager-Group Security (Data And Software)
Engineering Executive Manager-Group Security (Data And Software)

Commbank • Sydney

Hybrid
AUD 180,000 - 240,000
Principal Engineer - Cloud & Platform Engineering
Principal Engineer - Cloud & Platform Engineering

Westpac Bank • Sydney

On-site
AUD 240,000 - 360,000
Special offers on banking products and
Flexible work arrangements
Learning and development opportunities
+1
Principal Engineer Data and AI
Principal Engineer Data and AI

Westpac Group • Sydney

Hybrid
AUD 180,000 - 260,000
Flexible work
Leave options
Learning & development
+2
Top locations