Senior Analyst, Applied AI & Automation | Sydney, AU

eFinancialCareers Ltd.; Sydney; On-site; AUD 140,000 - 190,000; Full time

Job summary

Commonwealth Bank is seeking a Senior Analyst, Applied AI & Automation to design agentic AI solutions that transform fraud and scams QA. You’ll work with Fraud Analytics, QA, Product and Technology teams to build measurable automation using SQL, Python, LLMs and orchestration frameworks.

This foundational role shapes how the capability evolves, bringing ideas to life with strong governance, reliability and human oversight across technical and operational stakeholders.

Qualifications

• Strong hands-on analytics capability with SQL and Python to extract, prepare, analyse and interpret complex data.
• Experience applying classification, predictive analytics, ML or statistical modelling to real-world problems.
• Experience designing solutions using Generative AI and LLMs, not just using AI tools.
• Experience developing agentic or multi-step AI workflows including orchestration and tool use.
• Experience applying NLP or conversation analytics to extract insights from unstructured text.
• Understanding AI evaluation, testing model/agent behaviour and reducing hallucinations.
• Familiarity with agentic/orchestration frameworks (Copilot Studio, LangGraph, LangChain, Semantic Kernel, AutoGen).
• Strong development practices (Git, APIs) and collaborative AI-assisted environments.
• Curiosity to prototype ideas rapidly and translate analytics into measurable outcomes.

Responsibilities

• Design and build agentic AI solutions that automate and enhance the Fraud and Scams QA lifecycle.
• Explore how LLMs and intelligent automation can replace or augment manual QA processes.
• Develop multi-step AI workflows that reason across data and interact with tools.
• Use SQL and Python to analyse complex customer, interaction and operational data.
• Apply NLP and ML to support evaluation and classification tasks.
• Prototype, evaluate and improve AI approaches, focusing on reliability.
• Design testing and evaluation frameworks to measure performance and reduce failure modes.
• Collaborate with Fraud Analytics, QA, Product, Technology and Risk teams to drive implementation.

Skills

SQL

Python

NLP

LLMs

Agentic AI

AI workflows

Tools

Copilot Studio

LangGraph

LangChain

Semantic Kernel

AutoGen

Job description

Senior Analyst, Applied AI & Automation

• Build intelligent automation with agentic AI
• Shape a new AI automation capability
• Transform Fraud and Scams quality assurance

See Yourself in the Team

The Fraud, Scams & Cyber Data, Analytics and AI team brings together data, analytics and emerging AI capabilities to help protect our customers and the Group from increasingly sophisticated threats. Working across complex customer, operational and risk environments, the team uses data and technology to generate deeper insights, strengthen decision-making and develop smarter, more scalable ways of detecting, preventing and responding to fraud and scams.

You’ll join the People Capability Automation team, an emerging capability focused on exploring how analytics, AI and intelligent automation can transform traditionally manual processes across Fraud and Scams. Reporting to the People Capability Automation Lead, you’ll be part of the foundational team helping shape how the capability evolves - bringing ideas, experimenting with new approaches and building solutions that combine strong analytical foundations with the possibilities created by Generative and agentic AI.

With an initial focus on Quality Assurance, you’ll work closely with specialists across Fraud Analytics, Operations, Technology, Risk and AI to explore how customer interactions, operational data and existing processes can be better analysed and automated. This is an opportunity to help build something from the ground up, applying emerging technology to real operational challenges while ensuring solutions remain measurable, reliable and appropriately governed.

Your Impact & Contribution

This is a unique role where you’ll combine analytics, data science and agentic AI to help transform quality assurance across fraud and scam operations. Working with communications and interaction data, call transcripts, case management information and operational datasets, you’ll explore how LLMs, intelligent agents and agentic workflows can transform traditionally manual QA processes, from classifying fraud and scam scenarios and assessing conversation quality through to evaluating process adherence and surfacing actionable insights.

Partnering with Fraud Analytics, Quality Assurance, Product and Technology teams, you’ll help shape how agentic AI can be applied to improve quality, identify risks and translate promising concepts into practical, scalable solution, You’ll combine strong foundations in analytics with hands‑on applied AI capability, using SQL and Python alongside LLMs, classification models, NLP and agentic orchestration frameworks to design workflows that can reason across information, interact with data and tools, and coordinate tasks across the QA lifecycle while maintaining appropriate human oversight.

You’ll also play an important role in determining whether these solutions genuinely work, developing evaluation approaches, testing model and agent behaviour, improving reliability, reducing hallucinations and translating AI outputs into clear, defensible recommendations for operational, risk and technology stakeholders.

In summary, you may expect to:

• Design and build agentic AI solutions that automate and enhance elements of the Fraud and Scams quality assurance lifecycle.
• Explore where LLMs, agents and intelligent automation can replace or augment manual QA processes and help shape the solution approach.
• Develop and orchestrate multi-step AI workflows that can reason across unstructured information, interact with data and tools, and coordinate tasks.
• Use SQL and Python to explore, prepare and analyse complex customer, interaction, case and operational data.
• Apply NLP, classification, predictive analytics and machine learning where appropriate to complement agentic solutions.
• Prototype and evaluate new approaches, determining what works before developing scalable and reusable solutions.
• Design testing and evaluation frameworks to measure performance, identify failure modes and improve the reliability of AI outputs.
• Partner with Fraud Analytics, Quality Assurance, Product, Technology and Risk specialists to take ideas from problem definition and experimentation through to implementation and continuous improvement.

Your Skills & Experience

You’ll be a curious and solution-oriented analytics professional who combines strong foundations in data and analytics with hands‑on Applied AI capability. You thrive working with ambiguous problems, experimenting with emerging technologies and finding practical ways to turn ideas into reliable, scalable solutions. The ideal applicant will be comfortable interrogating complex data to understand what is happening and why, and exploring how AI, automation and intelligent agents can transform the way that work is performed.

In summary, you’ll bring:

• Strong hands‑on analytics capability with SQL and Python, with experience extracting, preparing, analysing and interpreting complex structured and unstructured datasets.
• Strong foundations in analytics and data science, with experience applying techniques such as classification, predictive analytics, statistical modelling or machine learning to solve real-world problems.
• Practical experience designing and building solutions using Generative AI and LLMs, rather than solely consuming AI tools.
• Experience developing agentic or multi-step AI workflows, including orchestration, reasoning, retrieval, tool use or interaction between AI agents, data and systems.
• Experience applying NLP, conversation analytics or LLMs to classify, evaluate or extract meaningful insights from unstructured text.
• An understanding of AI evaluation and reliability, including testing model and agent behaviour, measuring performance, identifying failure modes and reducing hallucinations.
• Experience with Copilot Studio, LangGraph, LangChain, Semantic Kernel, AutoGen or similar agentic and orchestration frameworks would be highly valuable.
• Familiarity with modern development practices, including Git/version control, APIs and collaborative or AI‑assisted development environments.
• A strong innovation and solution‑design mindset, with the curiosity to challenge existing approaches, rapidly prototype ideas and determine where analytics, AI or automation can deliver a better outcome.
• Strong stakeholder and communication skills, with the ability to translate ambiguous business problems into technical solutions and explain complex analytical and AI concepts clearly to technical and non‑technical audiences.
• Experience in fraud, scams, financial crime, quality assurance, risk, compliance or controls testing would be beneficial, but the priority is strong analytics and Applied AI capability with the ability to build solutions that are reliable, measurable and fit for purpose.

Working With Us

At CommBank, we’re committed to creating an accessible, inclusive and respectful workplace. If you require support or adjustments, please let us know.

We welcome applications from people of all backgrounds and we’re particularly committed to making a positive difference for Aboriginal and/or Torres Strait Islander Peoples. For support, please contact 1800 989 696.

We’re aware of some accessibility issues on this site, particularly for screen reader users. We want to make finding your dream job as easy as possible, so if you require additional support please contact HR Direct on 1800 989 696.