Get a reply from this employer
Your resume and cover letter, tailored to exactly what they’re hiring for.
EPAM Systems, Inc. is seeking a Python AI Evaluation Engineer to design evaluation frameworks for LLM-based apps and agents. You will build deterministic evaluators and Python-based tools to measure quality across responses, tool calls, and complete workflows.
You will integrate these checks into CI/CD pipelines, work with OpenTelemetry traces, and collaborate with AI Engineers and Platform Engineers to embed quality into development processes. AWS Bedrock Guardrails experience is a plus.
EPAM Systems, Inc. is seeking a Python AI Evaluation Engineer to design evaluation frameworks for LLM-based apps and agents. You will build deterministic evaluators and Python-based tools to measure quality across responses, tool calls, and complete workflows.
You will integrate these checks into CI/CD pipelines, work with OpenTelemetry traces, and collaborate with AI Engineers and Platform Engineers to embed quality into development processes. AWS Bedrock Guardrails experience is a plus.