Senior GenAI Evaluation Scientist

Johnson & Johnson Innovative Medicine; Cornellà de Llobregat; Híbrido; EUR 55.000 - 88.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Annual bonus

Vacation days

Parental leave

Bereavement leave

Caregiver leave

Well-being reimbursement

Descripción de la vacante

Johnson & Johnson Innovative Medicine in Europe seeks a Senior Scientist for Generative AI Evaluation & Standards (EQS). You will design, build, and run evaluation methods to assess LLMs, RAG systems, and AI agents used in R&D, ensuring scientific defensibility and traceability across programs.

You will author rubrics, curate datasets, validate judges, and deliver quality readouts that inform release decisions.

Formación

• Master’s degree in AI/ML, Computer Science, Data Science, Computational Biology, Bioinformatics, Biomedical Engineering, Applied Mathematics, Biostatistics, or a related field required.
• PhD preferred.

Responsabilidades

• Design, build, and maintain automated evaluation pipelines for LLM quality, RAG performance, agent reliability, safety, and scientific accuracy.
• Author evaluation rubrics and scoring criteria, curate golden and synthetic datasets with domain experts, and maintain our registry of reusable evaluation assets.
• Validate AI judges against human expert agreement and run model, prompt, retriever, and agent benchmarks that produce standardized quality readouts.
• Analyze failure patterns — hallucination, unsupported claims, weak traceability — and turn findings into actionable recommendations.
• Develop therapeutic-area-specific evaluation criteria with scientific, clinical, and regulatory partners, refining them based on real-world feedback.
• Design evaluation methods for scientific reasoning, evidence synthesis, and hypothesis quality — where generic benchmarks fall short.
• Build evaluation tooling and reusable patterns that enable other teams to self-serve.

Conocimientos

AI/ML evaluation

Python

LLMs

Prompt engineering

RAG

CI/CD evaluation

Data analysis

Educación

Master’s degree in AI/ML, CS, Data Science

PhD preferred

Herramientas

LLM APIs

Vector databases

Embedding models

Evaluation harnesses

Descripción del empleo

Johnson & Johnson Innovative Medicine in Europe seeks a Senior Scientist for Generative AI Evaluation & Standards (EQS). You will design, build, and run evaluation methods to assess LLMs, RAG systems, and AI agents used in R&D, ensuring scientific defensibility and traceability across programs.

You will author rubrics, curate datasets, validate judges, and deliver quality readouts that inform release decisions.