Senior Ai Test Automation Engineer, Hibrido

Ust; Madrid; Híbrido; EUR 70.000 - 95.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Health care plan

Teleworking compensation

Flexible benefits program

Training platforms access

Career stability and progression

Descripción de la vacante

UST busca un Ingeniero Senior de Automatización de Pruebas AI para trabajar en Madrid, con enfoque en pruebas de automatización híbridas y arquitectura de marcos escalables. Se valorará experiencia en Playwright, pytest y entornos CI/CD.

El rol implica colaborar con equipos multifuncionales y asegurar trazabilidad de pruebas, además de gestionar entornos (DEV, INT, UAT) y reportes de calidad, manteniendo estándares de gobernanza en un entorno regulado.

Formación

• Experiencia probada en automatización de pruebas con Python.
• Diseño y propiedad de marcos de prueba, no solo scripts.
• Integración de automatización en pipelines CI/CD (GitLab preferente).
• Conocimiento de ejecución en paralelo, herramientas de reporting y gestión de flakiness.
• Comprensión de Git, código limpio y revisiones de código.
• Experiencia con Docker y conocimientos básicos de Kubernetes.
• Exposición a pruebas de IA/ML (LLMs, RAG) o gran motivación para aprender.
• Comprensión de APIs y microservicios; principios SAFe, Agile y DevOps.

Responsabilidades

• Definir y evolucionar la arquitectura de automatización de pruebas alineada con la estrategia de pruebas.
• Diseñar marcos y patrones de automatización escalables y reutilizables entre equipos.
• Analizar requisitos y asegurar la trazabilidad de pruebas durante ceremonias Agile.
• Automatizar casos de prueba (funcionales y no funcionales) basados en riesgo y cobertura.
• Implementar automatización usando frameworks de Python (Playwright, pytest, Behave/BDD).
• Desarrollar enfoques de pruebas AI/ML (evaluación de LLM, comportamientos no determinísticos, flujos RAG).
• Construir y mantener pruebas de contrato de API y validación de microservicios.
• Integrar y ejecutar pruebas automatizadas en pipelines CI/CD (GitLab).
• Usar Docker/Kubernetes para entornos de prueba reproducibles.
• Gestionar ejecución de pruebas entre entornos (DEV, INT, UAT, pre-PROD).
• Monitorear salud de pruebas, gestionar pruebas inestables y reportes confiables.
• Reportar defectos en Jira con trazabilidad y evidencia.
• Producir artefactos de pruebas auditables y cumplir con gobernanza.
• Colaborar con equipos cruzados (POs, Ingenieros, MLOps, Data Engineers).
• Actuar como Embajador de Automatización dentro del ART.
• Asegurar trazabilidad end-to-end entre requisitos, pruebas y resultados.
• Mantener y evolucionar suites de pruebas conforme cambian APIs, modelos y prompts.

Conocimientos

Python-based test automation

CI/CD integration

Test framework design

AI/ML testing exposure

Agile/SAFe knowledge

Herramientas

Playwright

pytest

Behave/BDD

GitLab

Docker

Kubernetes

Jira

Descripción del empleo

Senior AI Test Automation Engineer, hibrido UST Madrid, España

Descripción del trabajo

Senior AI Test Automation Engineer We are looking for the very Top Talent and we would be delighted if you were to join our team! More in detail, UST is a multinational company based in North America, certified as a Top Employer and Great Place to Work, with over 35,000 employees worldwide and presence in more than 35 countries. We are leaders in digital technology services, delivering large-scale technological solutions to top-tier companies.

Principal responsibilities

• Define and evolve the test automation architecture aligned with the Bank s Test Strategy
• Design scalable, reusable automation frameworks and patterns across teams
• Analyse requirements and ensure testability during Agile ceremonies (PI Planning, Backlog Refinement, etc.)
• Design and automate test cases (functional non-functional) based on risk and coverage
• Implement automation using Python frameworks (Playwright, pytest, Behave/BDD)
• Develop AI/ML-specific testing approaches (LLM evaluation, non-deterministic behaviour, RAG workflows)
• Build and maintain API contract tests and microservices validation
• Integrate and run automated tests in CI/CD pipelines (GitLab)
• Use Docker/Kubernetes for reproducible test environments
• Manage test execution across environments (DEV, INT, UAT, pre-PROD)
• Monitor test health, handle flaky tests, and ensure reliable reporting
• Report defects in Jira with proper traceability and evidence
• Produce audit-ready test artefacts and ensure compliance with governance standards
• Collaborate with cross-functional teams (POs, Engineers, MLOps, Data Engineers)
• Act as Test Automation Ambassador within the ART
• Ensure end-to-end traceability across requirements, tests, and results
• Maintain and evolve test suites as systems (APIs, models, prompts) change

What do we expect from you?

• Strong experience in Python-based test automation (Playwright, pytest, BDD frameworks)
• Proven ability to design and own test frameworks, not just test scripts
• Experience integrating automation into CI/CD pipelines (GitLab preferred)
• Knowledge of parallel execution, reporting tools, and flaky test management
• Solid understanding of Git, clean code, and code review practices
• Experience with Docker and basic knowledge of Kubernetes
• Exposure to AI/ML testing (LLMs, RAG, non-deterministic systems) or strong motivation to learn
• Understanding of APIs, microservices, event-driven architectures
• Familiarity with SAFe, Agile, and DevOps principles
• Experience with Jira and test management tools (Xray, Agile Hive, etc.)
• Strong analytical mindset with end-to-end system understanding
• Awareness of data privacy, security, and compliance in regulated environments

Work Location

Hybrid position in Madrid (3 days at clients office, Sol area, next to the underground).

Work Schedule

Business hours (40 hours weekly with 1 hour for lunch).

What can we offer?

• 23 days of Annual Leave plus the 24th and 31st of December as discretionary days!
• Numerous benefits (Health Care Plan, teleworking compensation, Life and Accident Insurances).
• Retribución Flexible Program: (Meals, Kinder Garden, Transport, online English lessons, Health Care Plan )
• Free access to several training platforms
• Professional stability and career plans
• compensates referrals from which you could benefit when you refer professionals.
• The option to pick between 12 or 14 payments along the year.
• Real Work Life Balance measures (flexibility, WFH or remote work policy, compacted hours during summertime )
• UST Club Platform discounts and gym Access discounts

Python, Playwright, pytest, Behave, BDD, GitLab CI,