Senior Ai Qa Automation Engineer

Encora Inc.; Comisión de Fomento de Perú; A distancia; ARS 138.361.000 - 199.855.000; Jornada completa

Descripción de la vacante

Encora Inc. busca un Ingeniero Senior de QA de Automatización de IA para diseñar marcos de evaluación automatizados de agentes de IA y soluciones RAG, garantizando la calidad de sistemas de IA en entornos de producción.

Se requieren 5+ años en QA/Automatización, experiencia con Playwright o Selenium, pruebas API/UI y pipelines CI/CD. Trabajo remoto internacional desde LATAM.

Formación

• 5+ años de experiencia en Ingeniería de Calidad de Software, Automatización de Pruebas o Ingeniería de Fiabilidad.
• Experiencia práctica con Playwright y/o Selenium.
• Fortes habilidades en pruebas API, UI, integración, regresión y CI/CD.
• Experiencia en métricas de evaluación, conjuntos de datos de referencia y umbrales de calidad.

Responsabilidades

• Construir marcos de evaluación automatizados para agentes de IA, soluciones RAG, prompts y salidas de modelos.
• Probar conductas de IA como alucinaciones, grounding, precisión de citas y ataques de prompts.
• Desarrollar soluciones de QA impulsadas por agentes para generación de pruebas, validación de API y flujos de trabajo.
• Implementar puertas de calidad CI/CD y monitorear calidad de IA y fallos en producción.

Conocimientos

Python

Test Automation

API Testing

UI Testing

End-to-end Testing

CI/CD

QA Metrics

Educación

Bachelor’s degree in Computer Science / Software Engineering / Information Systems

Herramientas

Playwright

Selenium

Azure AI Foundry Evaluations

Descripción del empleo

Senior AI QA Automation Engineer

Empresa: Encora Inc. Tipo de empleo: Tiempo completo Remote / Work from Home

Descripción del trabajo - Senior AI QA Automation Engineer

Job Title: Senior AI QA Automation Engineer

Key Skills: Python, AI Testing, Test Automation, Playwright, Selenium, API Testing, Azure AI Foundry Evaluations, RAG Evaluation, Prompt Testing, CI/CD, Monitoring

Experience: 5+ YOE.

Location: Costa Rica, Peru, Colombia, Bolivia.

Mode: Remote

We at Coforge are hiring Senior AI QA Automation Engineer (#006fu00000bWsOCAA0-4) with the following skill set.

Key Responsibilities

• Build automated evaluation frameworks for AI agents, RAG solutions, prompts, tools, workflows, and model outputs, including benchmark datasets, golden responses, regression suites, evaluation metrics, and release thresholds.

• Test AI-specific behaviors including hallucinations, grounding, citation accuracy, prompt injection, tool selection, and multi-agent coordination.

• Develop agent-powered QA solutions for test-case generation, API and workflow validation, and intelligent UAT, while building UI, API, integration, performance, and end-to-end automation.

• Implement CI/CD quality gates and continuously monitor production AI quality, drift, reliability, and recurring failure patterns.

Required Skills & Qualifications

• 5+ years of experience in Software Quality Engineering, Test Automation, or Reliability Engineering.

• Strong Python skills with experience building reusable automated testing frameworks.

• Hands-on experience with Playwright and/or Selenium.

• Strong experience with API, UI, integration, regression, end-to-end, and CI/CD-based test automation.

• Experience with Azure AI Foundry Evaluations, RAG evaluation, and prompt testing.

• Experience defining evaluation metrics, benchmark datasets, regression suites, and measurable quality thresholds.

• Knowledge of AI quality concepts such as hallucination testing, grounding, citation accuracy, prompt injection, and agent behavior validation.

• Experience implementing CI/CD quality gates and production monitoring.

• Ability to translate business requirements, workflows, and AI behaviors into measurable quality criteria.

• Bachelor’s degree in Computer Science, Software Engineering, Information Systems, or a related field.

Preferred Skills:

• Experience testing Generative AI, RAG solutions, copilots, or AI agents.

• Experience with LLM evaluation frameworks and advanced prompt testing.

• Experience developing agent-powered test generation, workflow validation, or intelligent UAT assistants.

• Experience with adversarial testing and Responsible AI testing.

• Experience monitoring AI quality, drift, reliability, and failure patterns in production environments.

Posted On: 12-10-2026

At Coforge, we hire professionals based solely on their skills and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.

Sobre la empresa

Encora Inc.

As Encora Inc. expands its footprint in Latin America, its acquisition of Nearsoft provides our clients with a unique chance to Nearshore on a global scale.