Senior GenAI Evaluation Scientist

Johnson & Johnson Innovation; Madrid; Híbrido; EUR 55.000 - 88.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Annual bonus

Vacation days

Parental leave

Descripción de la vacante

Johnson & Johnson is seeking a Senior Scientist in Generative AI Evaluation to design, build, and run evaluation methods across R&D. You will author rubrics, curate datasets, validate AI judges, and provide quality readouts for release decisions.

This hybrid role is based in Madrid or Barcelona, with limited travel within Europe. You will work with multidisciplinary teams to develop domain-specific evaluation criteria, focusing on scientific accuracy, traceability, and regulatory considerations,

Formación

• We are seeking 6+ years in AI/ML evaluation or data science (Master's) or 3+ years with a PhD.
• Experience designing evaluation frameworks, benchmarks, or quality assessments for AI/ML systems.
• Hands-on work with generative AI: LLMs, RAG, agents, and prompt engineering.
• Strong ability to translate expert judgment into measurable criteria and reproducible protocols.

Responsabilidades

• Design, build, and run evaluation pipelines for LLM quality, RAG, agents, safety, and scientific accuracy.
• Create rubrics, curate datasets, and maintain a registry of reusable evaluation assets.
• Validate AI judges and run model, prompt, retriever, and agent benchmarks for standardized readouts.
• Analyze failure patterns and turn findings into actionable recommendations.
• Develop therapeutic-area-specific evaluation criteria with scientific partners.
• Design evaluation methods for scientific reasoning and evidence synthesis.
• Build tooling and patterns for self-serve evaluation by other teams.

Conocimientos

AI/ML evaluation

Data Science

Prompt engineering

Python

LLM APIs

Educación

Master's degree in AI/ML, CS, Data Science, Computational Biology, Bioinformatics, Biomedical Engineering, Applied Math, Biostatistics, or related field

Herramientas

Evaluation harnesses

Embedding models

Vector databases

LLM APIs

Descripción del empleo

Johnson & Johnson is seeking a Senior Scientist in Generative AI Evaluation to design, build, and run evaluation methods across R&D. You will author rubrics, curate datasets, validate AI judges, and provide quality readouts for release decisions.

This hybrid role is based in Madrid or Barcelona, with limited travel within Europe. You will work with multidisciplinary teams to develop domain-specific evaluation criteria, focusing on scientific accuracy, traceability, and regulatory considerations,