Senior GenAI QA Engineer & Test Architect

Johnson Johnson; Cornellà de Llobregat; On-site; EUR 70,000 - 90,000; Full time

Job summary

Jobtailor in Cataluña is seeking a QA engineer focused on GenAI to design and implement automated and manual QA strategies for GenAI systems, with strong Python skills and experience with LLM-based testing.

You will build regression tests, generate synthetic data, validate prompts and model behavior, execute tests, debug pipelines, and support deployment while leading design reviews and coordinating with data scientists and developers.

Qualifications

• 3+ years of software engineering experience in production systems.
• Experience with GenAI systems and RAG/graph frameworks.
• Strong Python and API testing, data validation, and pipelines.
• Familiarity with LLM APIs and cloud platforms (AWS/GCP/Azure).

Responsibilities

• Design and implement QA strategies for GenAI systems.
• Build regression tests and synthetic data workflows.
• Validate prompt logic, model behavior and metrics.
• Execute tests, debug pipelines, and support deployment.
• Lead design reviews and establish CI/CD testing standards.
• Collaborate with data scientists and developers; conduct code reviews.

Skills

Python

LLM-based systems

RAG frameworks

APIs testing

Data validation

CI/CD testing

Tools

Git

AWS Bedrock

Azure Foundry

GCP

Test harnesses

Job description

Jobtailor in Cataluña is seeking a QA engineer focused on GenAI to design and implement automated and manual QA strategies for GenAI systems, with strong Python skills and experience with LLM-based testing.

You will build regression tests, generate synthetic data, validate prompts and model behavior, execute tests, debug pipelines, and support deployment while leading design reviews and coordinating with data scientists and developers.