AI Quality Engineer: Production-Ready AI Testing (Hybrid)

Aviva plc; Województwo mazowieckie; Hybrid; PLN 180,000 - 260,000; Full time

Benefits offered by this job

Performance Bonus

Private medical care

Cafeteria benefits

Hybrid working from Poland

Job summary

Aviva plc is seeking an AI Quality Engineer for its Services Excellence Centre (ASEC) in Poland. The role focuses on evaluating AI-powered systems to ensure reliability, safety and production readiness across conversational AI and multi-turn contexts.

You will translate business objectives into measurable evaluation criteria, design robust testing approaches, build evaluation datasets and communicate findings to engineers, product teams and stakeholders.

Qualifications

• Professional experience in quality engineering, software testing or related field.
• Experience or strong interest in AI-powered solutions and evaluation.
• Good Python proficiency for analysis, automation or testing.
• Strong analytical and problem-solving skills.
• Ability to translate objectives and risks into evaluation approaches.
• Strong communication with technical and non-technical stakeholders.

Responsibilities

• Translate business expectations and risks into measurable evaluation criteria.
• Define successful, partially successful and unsuccessful system behaviour.
• Design evaluation approaches for conversational, multi-turn AI systems.
• Build and maintain representative evaluation datasets.
• Assess dataset quality, coverage and freshness.
• Select appropriate evaluation methods (manual, model-based, statistical).
• Analyze errors by category and root cause.
• Communicate results, uncertainty and risks to stakeholders.

Skills

Python

Quality engineering

Test automation

Data analysis

AI understanding

Communication skills

Education

Bachelor's degree in CS or related field

Tools

Git

Job description

Aviva plc is seeking an AI Quality Engineer for its Services Excellence Centre (ASEC) in Poland. The role focuses on evaluating AI-powered systems to ensure reliability, safety and production readiness across conversational AI and multi-turn contexts.

You will translate business objectives into measurable evaluation criteria, design robust testing approaches, build evaluation datasets and communicate findings to engineers, product teams and stakeholders.