Senior GenAI Evaluation Scientist: AI Quality & Standards

Johnson & Johnson Innovative Medicine; Madrid; Híbrido; EUR 55.000 - 88.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Annual bonus

Vacation days

Parental leave

Descripción de la vacante

Johnson & Johnson is seeking a Senior Scientist in Generative AI Evaluation & Standards to design, build, and run evaluation methods across R&D. You will author rubrics, curate datasets, validate AI judges, and produce quality readouts guiding release decisions.

Hybrid role in Madrid or Barcelona with limited travel, 6+ years in AI/ML evaluation, strong Python skills, and experience with LLMs and RAG are preferred.

Formación

• Master's degree in AI/ML, Computer Science, Data Science, Computational Biology, Biomedical Engineering, or a related field; PhD preferred.
• 6+ years of hands-on experience in AI/ML evaluation or data science (Master's) or 3+ years with PhD.

Responsabilidades

• Design, build, and maintain automated evaluation pipelines for AI/ML systems across R&D.
• Author evaluation rubrics and scoring criteria, curate datasets, and maintain the registry of reusable evaluation assets.
• Validate AI judges against human expert agreement and run model, prompt, retriever, and agent benchmarks.
• Analyze failure patterns and translate findings into actionable recommendations.
• Develop evaluation criteria with scientific, clinical, and regulatory partners.
• Design evaluation methods for scientific reasoning, evidence synthesis, and hypothesis quality.
• Build evaluation tooling and reusable patterns for self-serve by other teams.

Conocimientos

Python

AI/ML evaluation

Data analysis

Collaboration

Educación

Master's degree

PhD preferred

Herramientas

LLM APIs

Vector databases

Embedding models

Evaluation harnesses

Descripción del empleo

Johnson & Johnson is seeking a Senior Scientist in Generative AI Evaluation & Standards to design, build, and run evaluation methods across R&D. You will author rubrics, curate datasets, validate AI judges, and produce quality readouts guiding release decisions.

Hybrid role in Madrid or Barcelona with limited travel, 6+ years in AI/ML evaluation, strong Python skills, and experience with LLMs and RAG are preferred.