Senior Quality Assurance Engineer (AI)

Automic Group; Sydney; On-site; AUD 150,000 - 190,000; Full time

Benefits offered by this job

Career growth

Supportive environment

Growth opportunities

Job summary

Automic Group is seeking a Senior Quality Assurance Engineer – AI to drive independent quality assurance for AI-enabled products. You will shape quality strategy with the AI Engineering team, establish evaluation datasets, and ensure safe, reliable delivery of AI solutions across our platforms.

You will lead risk-based testing, implement automated quality controls, and provide evidence-based release recommendations while collaborating with cross-functional teams in a dynamic fintech environment.

Qualifications

• Proven experience in quality engineering or test automation for complex production systems.
• Practical experience testing AI, ML, generative AI, agentic or retrieval-based applications.
• Strong capability in AI quality assurance, evaluation and risk-based testing.
• Programming experience in Python or another language used for test automation and service testing.
• Experience across automated service, integration, security, resilience and non-functional testing.
• Strong analytical skills and the ability to make evidence-led quality and release decisions.
• Excellent communication skills with the ability to influence and coach cross-functional delivery teams.

Responsibilities

• Lead risk-based quality strategies for AI applications, agents, retrieval systems and supporting services.
• Deliver functional and non-functional assurance across integrations and end-to-end workflows.
• Establish AI evaluation datasets, test data, quality thresholds and automated regression coverage.
• Assess AI response quality and safety, including hallucination, groundedness, adversarial behaviour and sensitive-data exposure.
• Validate agent workflows, authentication, authorisation, human approvals and tenant isolation.
• Drive automated quality controls, defect management and continuous improvement across delivery pipelines.
• Provide evidence-based release recommendations and investigate production quality issues.

Skills

Quality engineering

Test automation

AI testing

Python

Non-functional testing

Risk-based testing

Communication

Education

Tech/Engineering degree

Job description

AutomicGroup is Australia’s market leader in technology-driven investor administration. Combining proprietary cloud-native technology with expert service,Automicsimplifies investor administration and governance for companies and funds. We provide the market's leading registry, employee share plans, IR analytics, and fund administration services for over 1,400 ASX-listed and unlisted organisations.

Automic’sculture is collaborative, inclusive, and purpose driven. It is grounded in our mission to connect companies and funds to their investors through technology, service, and trust, and guided by our values ofOneAutomic, Go Beyond, Empowered, and Accountable.

About the Role

We'relooking for aSenior Quality Assurance Engineer – AIto provide independent assurance thatAutomic’sAI-enabled products, agents and platforms are reliable,secureand production-ready.

Working closely with our AI Engineering team,you’llshape quality strategy,establishrobust AI evaluation and testing practices, and provide evidence-based release and residual-risk advice.You’llplay a key role in ensuring our AI solutions deliver safe,reliableand high-quality outcomes.

Key Responsibilitiesinclude:

• Lead risk-based quality strategies for AI applications, agents, retrievalsystemsand supporting services.
• Deliver functional and non-functional assurance across integrations and end-to-end workflows.
• Establish AI evaluation datasets, test data, qualitythresholdsand automated regression coverage.
• Assess AI response quality and safety, including hallucination,groundedness, adversarial behaviour and sensitive-data exposure.
• Validate agent workflows, authentication, authorisation, humanapprovalsand tenant isolation.
• Drive automated quality controls, defectmanagementand continuous improvement across delivery pipelines.
• Provide evidence-based release recommendations and investigate production quality issues.

About you!

To be successful in this role,you will have:

Proven experience in quality engineering or test automation for complex production systems.

Practical experience testing AI, machine learning, generative AI, agentic or retrieval-based applications.

Strong capability in AI quality assurance,evaluationand risk-based testing.

Programming experience in Python or another language used for test automation and service testing.

Experience across automated service, integration, security,resilienceand non-functional testing.

Strong analytical skills and the ability to make evidence-led quality and release decisions.

Excellent communication skills, with the ability to influence and coach cross-functional delivery teams.

A tertiary qualification in technology, engineering or a related discipline, or equivalent professional experience.

Experience in financial services or another regulated environment is desirable.

You’llbe part of a team whereoperational excellence, collaboration, and continuous improvementare genuinely valued. This is an opportunity to deepen yourexpertise, contribute to meaningful outcomes, and grow your career within a high-performing, purpose-led fintech environment.

Key Benefits include:

• Professional career growth & development
• Supportive & fast paced environment
• Opportunity to grow with the business