AI Assurance & Evaluation Specialist

Profectus; Canberra; On-site; AUD 150,000 - 190,000; Full time

Job summary

Profectus is seeking an experienced AI Test and Evaluation and Assurance Specialist to support the safe adoption of AI-enabled capability within Australian Government contexts from Canberra. The role provides independent, evidence-based assessment of AI-enabled systems, governance artefacts and advisory support for risk and suitability at senior levels.

The successful candidate will plan and oversee AI assurance activities, review technical papers, and translate findings into clear reports and

Qualifications

• Tertiary STEM qualification or equivalent professional experience.
• Minimum five years experience in AI/ML/advanced technology governance or assurance.
• Experience applying systems engineering, risk, assurance and test methods to real-world problems.
• Experience planning or overseeing assurance, verification, validation and evaluation activities.
• Strong understanding of AI/ML systems and ability to assess documentation, performance claims and risks.
• Experience developing governance, policy, standards or risk management in tech settings.
• Ability to interpret policy/governance requirements and translate into practical implementation.
• Experience preparing reports for senior stakeholders and decision makers.
• Experience engaging with cross-functional stakeholder groups.
• Postgraduate qualification in AI/ML/Data Science/Systems Engineering/operational analysis.
• Experience delivering AI/ML/data analytics within Commonwealth contexts.
• Experience with generative AI/LLMs/RAG/CV/autonomy or advanced analytics.
• Experience supporting verification/validation/assurance/certification or accreditation.
• Experience acting as customer representative or technical authority.
• Experience assessing vendor claims and solution effectiveness.
• Experience implementing AI governance and responsible AI approaches.
• Experience with cloud AI platforms (Azure/AWS/GCP).
• Experience with AI/ML tools (TensorFlow, PyTorch, Scikit-learn); proficient in Python.
• Relevant industry certifications in AI/ML/cloud AI, systems engineering.

Responsibilities

• Support AI assurance and test across AI-enabled systems.
• Assess technical evidence, performance claims, limitations and risks.
• Coordinate evaluation activities delivered by industry and researchers.
• Advise senior stakeholders on risk and suitability.
• Develop AI governance artefacts including use cases and model cards.
• Translate technical findings into actionable reports for decision makers.
• Capture and transfer knowledge to support workforce continuity.

Skills

AI governance

Risk management

Stakeholder engagement

Technical reporting

Cloud platforms

Python

Education

Postgraduate AI/Data Science/Systems Engineering

Bachelor’s degree in STEM

Tools

TensorFlow

PyTorch

Scikit-learn

Azure/AWS/Google Cloud

Job description

Profectus is seeking an experienced AI Test and Evaluation and Assurance Specialist to support the safe adoption of AI-enabled capability within Australian Government contexts from Canberra. The role provides independent, evidence-based assessment of AI-enabled systems, governance artefacts and advisory support for risk and suitability at senior levels.

The successful candidate will plan and oversee AI assurance activities, review technical papers, and translate findings into clear reports and