Senior QA Engineer for ML/AI Evaluation Platforms (Remote)

EPAM Systems, Inc.; Polska; Hybrid; PLN 180,000 - 240,000; Full time

Benefits offered by this job

Health insurance

Multisport

Relocation support

Remote work within Poland

Job summary

EPAM Systems, Inc. is seeking a Senior QA / ML Tester to own quality assurance for AI Platform, focusing on Agent Evaluation Framework built on AWS AgentCore.

You will design, implement, and maintain a functional test suite validating AI evaluation pipelines and providing feasibility assessments for non-AgentCore runtimes. In a production, Agile setting, you will collaborate with platform engineers, contribute to CI/CD pipelines, and ensure trace-based evaluation quality using OpenTelemetry.

Qualifications

• 5+ years of production QA automation or ML/AI system testing.

Responsibilities

• Design and implement a functional test suite for the AWS AgentCore Evaluation API using pytest, covering known-good / known-bad session pair validation, multi-evaluator execution correctness, and edge case handling.

Skills

QA automation

ML system testing

Python testing

OpenTelemetry testing

REST API testing

CI/CD pipelines

Jira

Problem solving

Tools

pytest

OpenTelemetry

unittest.mock

Moto

AWS testing

Job description

EPAM Systems, Inc. is seeking a Senior QA / ML Tester to own quality assurance for AI Platform, focusing on Agent Evaluation Framework built on AWS AgentCore.

You will design, implement, and maintain a functional test suite validating AI evaluation pipelines and providing feasibility assessments for non-AgentCore runtimes. In a production, Agile setting, you will collaborate with platform engineers, contribute to CI/CD pipelines, and ensure trace-based evaluation quality using OpenTelemetry.