Senior AI Quality Engineer — SaaS Evaluation & Reliability

StarRez; Australia; Hybrid; AUD 140,000 - 190,000; Full time

Benefits offered by this job

Weekly team lunches

Hybrid workplace

Professional development opportunities

Global team collaboration

Job summary

StarRez, the global leader in student housing software, is seeking a Senior Quality Engineer - AI to shape how we evaluate, validate, and improve AI-powered product experiences. You will balance hands-on work with mentoring the team and influence product readiness across LLM, RAG, prompts, and tool-use.

You’ll partner with engineers, product managers, and SMEs to raise the bar on accuracy, safety, and reliability, build repeatable evaluation systems, and translate feedback into concrete product

Qualifications

• Proven senior individual contributor with quality focus.
• Data-driven approach using metrics and dashboards.
• Ability to guide engineers, product partners, and stakeholders.

Responsibilities

• Lead or contribute to AI quality and evaluation strategy for LLMs, RAG, prompts, tools, and agent workflows.
• Provide input on testability, observability, scalability, and reliability in feature discussions.
• Coach reviewers and SMEs; mentor cross-functional partners on evaluation-driven development.

Skills

Senior IC

Quality mindset

Data-driven

Technical communication

Tools

Test automation

Observability

Scripting

Evaluation tooling

Job description

StarRez, the global leader in student housing software, is seeking a Senior Quality Engineer - AI to shape how we evaluate, validate, and improve AI-powered product experiences. You will balance hands-on work with mentoring the team and influence product readiness across LLM, RAG, prompts, and tool-use.

You’ll partner with engineers, product managers, and SMEs to raise the bar on accuracy, safety, and reliability, build repeatable evaluation systems, and translate feedback into concrete product