Senior GenAI Quality Engineer – Testing & Reliability

Johnson Johnson; Madrid; On-site; EUR 55,000 - 90,000; Full time

Job summary

Jobtailor seeks a seasoned software engineer/QA specialist to design and execute automated and manual QA strategies for GenAI systems in Madrid. You will build regression tests, synthetic data, and evaluation pipelines to verify accuracy, grounding, robustness, and safety.

You will validate prompt logic and model behavior, run tests, review code, and contribute to deployment of GenAI products. Collaboration with data scientists and developers will be essential to embed testing checkpoints and

Qualifications

• Strong software engineering background with QA ownership.
• 3+ years as a software engineer or QA in production systems.
• Advanced Python skills and experience with GenAI systems.
• Proven ability to implement test frameworks for GenAI applications.
• Experience with API testing, data validation, and data pipelines.

Responsibilities

• Design and implement automated and manual QA strategies for GenAI systems.
• Build regression tests, synthetic data, and evaluation pipelines for accuracy, grounding, robustness, and safety.
• Validate prompt logic and model behavior; execute tests and debug pipelines.
• Support deployment of GenAI products and lead design reviews for reliability.
• Collaborate with data scientists and developers to embed testing checkpoints and CI/CD workflows.

Skills

Python

GenAI QA

LLM testing

Test framework design

Data validation

Problem solving

Education

Bachelor's degree in CS/Engineering

Tools

LLM APIs

Git

Cloud platforms (AWS, Azure, GCP)

CI/CD

Job description

Jobtailor seeks a seasoned software engineer/QA specialist to design and execute automated and manual QA strategies for GenAI systems in Madrid. You will build regression tests, synthetic data, and evaluation pipelines to verify accuracy, grounding, robustness, and safety.

You will validate prompt logic and model behavior, run tests, review code, and contribute to deployment of GenAI products. Collaboration with data scientists and developers will be essential to embed testing checkpoints and