Senior GenAI Evaluation Scientist — R&D Quality & Standards

Johnson & Johnson Co.; Madrid; Híbrido; EUR 55.000 - 88.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Annual bonus

Vacation days

Parental leave

Bereavement leave

Well-being reimbursement

Descripción de la vacante

Johnson & Johnson is seeking a Senior Scientist to lead the evaluation of Generative AI systems within the EQS team in Data Science and Digital Health. You will design, build, and run evaluation methods to ensure AI readiness for real scientific work across R&D projects.

You will author rubrics, curate datasets with domain experts, validate AI judges, and produce quality readouts to guide release decisions, collaborating with regulatory partners and cross-disciplinary teams.

Formación

• Master's degree in AI/ML, Computer Science, Data Science, Computational Biology, Bioinformatics, Biomedical Engineering, Applied Mathematics, Biostatistics, or a related field required.
• PhD preferred.

Responsabilidades

• Design, build, and maintain automated evaluation pipelines for LLM quality, RAG performance, agent reliability, safety, and scientific accuracy.
• Author evaluation rubrics and scoring criteria, curate golden and synthetic datasets, validate AI judges, and produce quality readouts for release decisions.
• Analyze failure patterns and turn findings into actionable recommendations.
• Develop evaluation criteria with scientific, clinical, and regulatory partners, refining them with real-world feedback.
• Design evaluation methods for scientific reasoning, evidence synthesis, and hypothesis quality.
• Build evaluation tooling and reusable patterns enabling cross-team self-serve.

Conocimientos

Python

AI/ML evaluation

Evaluation frameworks

Generative AI

Prompt engineering

Collaboration

Data Visualization

Data Science

CI/CD pipelines

Regulatory environments

Educación

Master's degree in AI/ML, CS, Data Science or related

PhD preferred

Herramientas

Embedding models

Vector databases

LLM APIs

Evaluation harnesses

CI/CD pipelines for evaluation

Descripción del empleo

Johnson & Johnson is seeking a Senior Scientist to lead the evaluation of Generative AI systems within the EQS team in Data Science and Digital Health. You will design, build, and run evaluation methods to ensure AI readiness for real scientific work across R&D projects.

You will author rubrics, curate datasets with domain experts, validate AI judges, and produce quality readouts to guide release decisions, collaborating with regulatory partners and cross-disciplinary teams.