Senior GenAI Evaluation Scientist for Pharma R&D

7300-Janssen-Cilag S.A. Legal Entity; Madrid; Híbrido; EUR 55.000 - 88.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Annual bonus

Vacation days

Descripción de la vacante

Johnson & Johnson is seeking a Senior Scientist to design, build, and run evaluation methods for generative AI systems in pharmaceutical R&D. You will author rubrics, curate benchmarks, and validate AI judges across models, prompts, and agents.

The role focuses on quality, traceability, and scientific defensibility, collaborating with clinical and regulatory partners to shape release decisions. Hybrid role based in Europe with travel.

Formación

• Master's degree in AI/ML, Computer Science, Data Science or related field required.
• PhD preferred for advanced research focus and leadership.
• Minimum 6+ years hands-on AI/ML evaluation or 3+ years with PhD.

Responsabilidades

• Design, build, and maintain automated evaluation pipelines for LLM quality and RAG performance.
• Author rubrics and scoring criteria; curate datasets and maintain evaluation assets.
• Validate AI judges against human expert agreement; run benchmarks across models and prompts.
• Analyze failure patterns and provide actionable recommendations.
• Develop domain-specific evaluation criteria with scientific and regulatory partners.
• Design evaluation methods for scientific reasoning and evidence synthesis.

Conocimientos

Python proficiency

AI/ML evaluation

Rubrics & benchmarks

Team collaboration

English proficiency

Data analysis

Vector databases

LLM APIs

Educación

Master's degree in AI/ML or related field

PhD preferred

Herramientas

Evaluation harnesses

Embedding models

CI/CD for evaluation

Descripción del empleo

Johnson & Johnson is seeking a Senior Scientist to design, build, and run evaluation methods for generative AI systems in pharmaceutical R&D. You will author rubrics, curate benchmarks, and validate AI judges across models, prompts, and agents.

The role focuses on quality, traceability, and scientific defensibility, collaborating with clinical and regulatory partners to shape release decisions. Hybrid role based in Europe with travel.