Postdoc: GenAI Evaluation & Standards for Pharma

Johnson & Johnson Innovative Medicine; Madrid; Presencial; EUR 44.000 - 70.000; Jornada completa

Descripción de la vacante

Johnson & Johnson Innovative Medicine seeks a postdoctoral researcher to design methods testing GenAI tools for accuracy, evidence grounding, traceability, and usability in pharmaceutical R&D. You will collaborate with scientific, clinical, regulatory, and data science teams to establish evaluation benchmarks and release criteria.

The role focuses on developing criteria, datasets, and annotation protocols, translating expert judgment into measurable outcomes, and advocating for responsible AI

Formación

• Strong knowledge of biomedical science and pharmaceutical R&D.
• Experience translating expert judgment into criteria, rubrics, datasets, protocols, or measurable outcomes.
• Experience designing or applying evaluation methods, benchmark datasets, annotation protocols, validation studies, quality reviews, or assessment frameworks.
• Interest in testing GenAI systems, including LLMs, RAG, and AI agents.
• Proficiency in Python and common data science or machine learning tools.
• Clear written and verbal communication skills.

Responsabilidades

• Design evaluation frameworks, rubrics, and criteria for GenAI tools used across pharmaceutical R&D.
• Develop therapeutic-area-specific criteria with business and scientific teams to reflect domain and use-case quality needs.
• Build benchmark datasets, reference answer sets, annotation guides, and evaluation datasets.
• Run expert reviews with scientific, clinical, regulatory, medical, data science, and engineering teams.
• Test LLM, RAG, and agent performance, including accuracy, source grounding, retrieval quality, citation fidelity, task completion, robustness, safety, and usability.
• Analyze failure patterns such as unsupported claims, incorrect reasoning, poor evidence use, missing uncertainty, weak traceability, or failure to follow instructions.
• Translate evaluation findings into improvements in prompts, retrieval methods, agent workflows, tools, and user experience.
• Help define release criteria for systems moving from prototype to limited release, expanded use, or product support.
• Review emerging evaluation methods and adapt useful approaches for pharmaceutical R&D.
• Document methods, findings, and recommendations so teams can apply consistent evaluation practices.
• Design and develop agentic judge methods to evaluate GenAI outputs against defined criteria, flag evidence gaps or unsupported claims, and support expert review workflows.

Conocimientos

Biomedical research

Python

Data science

Benchmarking

Rubric design

Technical communication

Educación

PhD or equivalent in biomedical science, computational biology, AI/ML, data science, clinical research, regulatory science, biostatistics, pharmaceutical sciences, or related field

Herramientas

LLM APIs

Embeddings

Vector databases

Prompt engineering

Agent frameworks

AI evaluation tools

Descripción del empleo

Johnson & Johnson Innovative Medicine seeks a postdoctoral researcher to design methods testing GenAI tools for accuracy, evidence grounding, traceability, and usability in pharmaceutical R&D. You will collaborate with scientific, clinical, regulatory, and data science teams to establish evaluation benchmarks and release criteria.

The role focuses on developing criteria, datasets, and annotation protocols, translating expert judgment into measurable outcomes, and advocating for responsible AI