AI Quality Engineer — Hybrid & Impactful AI

Aviva; Polska; Hybride; PLN 200 000 - 280 000; Plein temps

Avantages offerts par ce poste

Performance Bonus

Private medical care

MultiSport card

Hybrid working model from Poland with少

Résumé du poste

Aviva Services Excellence Centre in Poland is hiring an AI Quality Engineer to ensure quality, safety and reliability of AI-powered products. You will translate business outcomes into measurable evaluation criteria and design evaluation approaches for conversational AI systems.

You will build evaluation datasets, assess data quality, validate evaluators, and collaborate with engineers, product teams and stakeholders to release robust AI solutions.

Qualifications

• Professional experience in quality engineering, software testing or related field.
• Interest in AI-powered solutions and ability to evaluate AI systems.
• Strong analytical and problem solving skills; able to explain findings.

Responsabilités

• Translate business expectations into measurable evaluation criteria.
• Define system behaviour success levels and評価 thresholds.
• Design evaluation approaches for conversational and multi-turn AI.
• Build and maintain representative evaluation datasets.
• Assess dataset quality and coverage; check for bias and freshness.
• Select manual, deterministic and model-based evaluation methods.
• Analyze errors by category, journey, risk level and root cause.
• Distinguish model failures from retrieval, prompts or data issues.
• Validate automated evaluators against human judgement.
• Create baselines, regression tests and controlled comparisons.
• Prioritize tests by probability and impact of failure.
• Communicate results and residual risks for release decisions.

Connaissances

Python proficiency

Quality engineering

Data science

Analytical problem solving

Communication skills

Outils

Git

JSON/logs analysis

Description du poste

Aviva Services Excellence Centre in Poland is hiring an AI Quality Engineer to ensure quality, safety and reliability of AI-powered products. You will translate business outcomes into measurable evaluation criteria and design evaluation approaches for conversational AI systems.

You will build evaluation datasets, assess data quality, validate evaluators, and collaborate with engineers, product teams and stakeholders to release robust AI solutions.