Senior GenAI Evaluation Scientist — Pharma R&D Impact

Johnson & Johnson Innovative Medicine; Cornellà de Llobregat; Híbrido; EUR 55.000 - 88.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Annual bonus

Vacation days

Parental leave

Bereavement leave

Well-being reimbursement

Insurance plans

Descripción de la vacante

Johnson & Johnson is building Generative AI solutions to support pharmaceutical R&D and decision workflows. This hybrid role in Madrid or Barcelona involves designing evaluation frameworks for AI quality, safety, and scientific accuracy, collaborating with clinical and regulatory partners across Europe.

You will lead automated pipelines, curate datasets with domain experts, and benchmark models, prompts, and retrievers to ensure measurable, traceable, and defensible AI outputs in a regulated

Formación

• Master's degree in AI/ML or related field; PhD preferred.
• 6+ years in AI/ML evaluation or data science (Master's) or 3+ years with a PhD.
• Experience designing evaluation frameworks and scientific benchmarks.
• Hands-on with generative AI, LLMs, RAG, and prompt engineering.
• Strong Python skills and familiarity with modern AI/ML tooling.

Responsabilidades

• Design, build, and maintain automated evaluation pipelines for LLM quality, RAG performance, agent reliability, safety, and scientific accuracy.
• Author evaluation rubrics and scoring criteria, curate datasets with domain experts, and maintain reusable evaluation assets.
• Validate AI judges against human expert agreement and run benchmarks for models, prompts, retrievers, and agents.
• Analyze failure patterns and turn findings into actionable recommendations.
• Develop therapeu­tic-area specific evaluation criteria with scientific, clinical, and regulatory partners.
• Design evaluation methods for scientific reasoning and evidence synthesis.
• Build evaluation tooling and reusable patterns for cross-team use.

Conocimientos

Evaluation frameworks

Prompt engineering

Generative AI

Python

AI/ML tooling

Educación

Master's degree in AI/ML, CS, Data Science, Computational Biology, Bioinformatics, Biomedical Eng.

PhD preferred

Herramientas

Python

LLM APIs

Vector databases

Evaluation harnesses

Descripción del empleo

Johnson & Johnson is building Generative AI solutions to support pharmaceutical R&D and decision workflows. This hybrid role in Madrid or Barcelona involves designing evaluation frameworks for AI quality, safety, and scientific accuracy, collaborating with clinical and regulatory partners across Europe.

You will lead automated pipelines, curate datasets with domain experts, and benchmark models, prompts, and retrievers to ensure measurable, traceable, and defensible AI outputs in a regulated