AI Evaluation & Quality Tester

Hiberus; Wrocław; Hybrid; PLN 75,000 - 110,000; Full time

Job summary

Hiberus in Poland is seeking an AI Tester / AI Evaluation Engineer to assess the quality and stability of AI, ML and GenAI solutions across diverse environments.

You will design test scenarios, create reference data, and evaluate model outputs for accuracy, completeness, and security while ensuring usability and compliance with requirements. Collaboration with AI engineers and security teams will be essential throughout solution acceptance.

Qualifications

• Minimum 3 years of experience in software testing or data-driven solutions.
• Knowledge of test case design and defect reporting.
• Basic understanding of ML or LLM models and non-deterministic systems testing.
• Ability to design test suites for evaluating model output quality.
• SQL and API testing experience with Postman.
• Ability to assess accuracy, completeness, source consistency, security and usability.

Responsibilities

• Prepare test scenarios for AI/ML/GenAI solutions.
• Design question sets, edge cases, and reference data for model evaluation.
• Test the quality of model outputs and stability of AI system behaviour.
• Perform regression testing after changes to the model, prompts, source data or configuration.
• Report defects, anomalies and quality-related risks.
• Collaborate with AI Engineers, analysts, business and security teams during solution acceptance.
• Document test results and prepare recommendations regarding AI system quality.

Skills

Software testing

Test case design

API testing

SQL

ML/LLM basics

Defect reporting

Non-deterministic testing

Tools

Postman

API testing tools

Job description

Hiberus in Poland is seeking an AI Tester / AI Evaluation Engineer to assess the quality and stability of AI, ML and GenAI solutions across diverse environments.

You will design test scenarios, create reference data, and evaluate model outputs for accuracy, completeness, and security while ensuring usability and compliance with requirements. Collaboration with AI engineers and security teams will be essential throughout solution acceptance.