Associate Director, Evaluation and Quality Standards

J&J Family of Companies; Cornellà de Llobregat; Presencial; EUR 75.000 - 129.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Annual bonus

Comprehensive benefits

Professional development

Descripción de la vacante

Johnson & Johnson seeks a senior leader to head the Evaluation & Standards for GenAI within the Data Science & Digital Health group. You will define quality metrics, govern AI evaluation across R&D, and drive cross‑functional collaboration with QMS, IT, and therapeutic teams.

In this role you will build pipelines, set gates for PoC to production, and communicate complex findings to senior stakeholders while shaping strategic decisions in a regulated pharmaceutical environment.

Formación

• Minimum 8 years of post-academic industry experience in pharma/biotech R&D.
• Hands‑on expertise with generative AI systems and related evaluation.
• Proven track record designing evaluation frameworks, benchmarks, or quality standards.

Responsabilidades

• Design and maintain evaluation frameworks for GenAI solutions across quality and safety.
• Lead vendor assessments and benchmarking to validate claims.
• Define production readiness and establish evaluation gates for product stages.
• Partner with cross-functional teams on data quality and governance for AI workloads.
• Build and mentor the GenAI Evaluation & Standards team.

Conocimientos

Generative AI systems

Evaluation frameworks

Team leadership

Stakeholder communication

Prompt engineering

RAG performance

Quality governance

Educación

PhD in computational biology / bioinformatics / data science / CS / AI/ML

Herramientas

LLMs

Retrieval-Augmented Generation

Agentic frameworks

Descripción del empleo

At Johnson & Johnson,we believe health is everything. Our strength in healthcare innovation empowers us to build aworld where complex diseases are prevented, treated, and cured,where treatments are smarter and less invasive, andsolutions are personal.Through our expertise in Innovative Medicine and MedTech, we are uniquely positioned to innovate across the full spectrum of healthcare solutions today to deliver the breakthroughs of tomorrow, and profoundly impact health for humanity.Learn more at jnj.com (https://www.jnj.com.) .

As guided by Our Credo, Johnson & Johnson is responsible to our employees who work with us throughout the world. We provide an inclusive work environment where each person is considered as an individual. At Johnson & Johnson, we respect the diversity and dignity of our employees and recognize their merit.

Job Function:

Data Analytics & Computational Sciences

Job Sub Function:

Data Science

Job Category:

People Leader

All Job Posting Locations:

Cornellà de Llobregat, Barcelona, Spain, Madrid, Spain

Job Description:

Evaluation & Quality Standards to join our Data Science and Digital Health team (DSDH). This is a newly created leadership role within the Generative AI organization, reporting directly to the Head of Generative AI.

The GenAI team is deploying AI across discovery, development, regulatory, and operations to create AI assistants and agentic systems across the therapeutic areas and functions. As we scale from proof-of-concept to enterprise deployment, we need a dedicated evaluation function that defines what quality means for generative AI in a regulated pharmaceutical R&D environment, communicates the metrics effectively, and enforces that standard across every platform, vendor, and capability we build or adopt. This role owns evaluation and governance for all generative AI work across R&D.

Key Responsibilities

Evaluation Framework Design

• Design and maintain evaluation frameworks for GenAI solutions spanning LLM quality, RAG performance, agent reliability, safety, and scientific accuracy.
• Develop therapeutic-area-specific evaluation criteria with business teams, reflecting domain-specific quality requirements per therapeutic area and application type.
• Define benchmarks for scientific validity, regulatory compliance, data quality, and operational reliability.
• Establish gold-standard validation datasets and automated evaluation pipelines.

Vendor and Solution Assessment

• Lead structured vendor assessments producing written Evaluation Reports
• Challenge vendor claims with independent benchmarking; distinguish genuine capability from demonstration performance.
• Assess integration complexity, total cost of ownership, regulatory posture, and vendor lock-in risk.

Governance and Standards

• Serve as technical lead for the Evaluation & Standards Board, a cross-functional governance body for GenAI quality.
• Present evaluation findings and recommendations to the GenAI Portfolio Steering Committee.
• Define what “production-ready” means for GenAI in a regulated R&D environment and enforce that standard.
• Set evaluation gates for product development stages: PoC, Limited Release, Scaled Product, and Product Ops.

Cross-Functional Collaboration

• Partner with QMS/MLOps on the handoff from pre-adoption evaluation to production quality monitoring.
• Work with therapeutic area teams on domain-specific evaluation standards.
• Collaborate with Data Strategy & Products on data quality as an input to the evaluation rubric.
• Engage JJIT architecture teams on platform-layer evaluation and security assessments.

Opinion Leadership and Capability Building

• Contribute to the weekly GenAI Outlook newsletter, specifically the evaluation implications of frontier AI developments.
• Build internal evaluation capability through training, documentation, and tooling for teams conducting assessments.
• Publish evaluation standards and rubrics as reference material for the broader organization.

Team Leadership

• Build and lead the GenAI Evaluation & Standards team
• Attract, develop, and retain top talent in AI evaluation, quality science, and governance.
• Establish a culture of scientific rigor and independent judgment within the team.

What This Role Is Not

• Not MLOps or production monitoring: the QMS function handles post-deployment operations.
• Not platform engineering.
• Not regulatory affairs or medical writing.
• Not vendor management or procurement: evaluation is independent of purchasing.

Key Qualifications

• Advanced degree (PhD strongly preferred) in computational biology, bioinformatics, data science, computer science, AI/ML, biomedical engineering, applied mathematics, or related discipline.
• Minimum 8 years of post-academic industry experience, with significant time in pharmaceutical or biotechnology R&D.
• Hands‑on expertise with generative AI systems: large language models, retrieval‑augmented generation, agentic frameworks, and prompt engineering.
• Demonstrated track record designing evaluation frameworks, scientific benchmarks, or quality standards for AI/ML systems.
• Strong people leadership experience, including building and managing technical teams in a matrixed organization.
• Excellent communication skills: ability to present complex technical findings clearly to non-technical senior stakeholders and defend unpopular conclusions.
• Strategic thinking capability: ability to connect technical evaluation to business impact and organizational decision-making.

Preferred Skills:

Advanced Analytics, Budget Management, Compliance Management, Critical Thinking, Data Analysis, Data Privacy Standards, Data Quality, Data Reporting, Data Savvy, Data Science, Data Visualization, Developing Others, Digital Fluency, Inclusive Leadership, Leadership, Program Management, Strategic Thinking, Succession Planning

The anticipated base pay range for this position is:

€75,000.00 - €129,260.00

Benefits:

In addition to base pay, we offer the following benefits*: an annual bonus with set target (% of pay) depending on pay grade / location, where the actual amount is based on the employees’ and companies’ performance of the previous calendar year, or sales commissions. Moreover, we offer vacation days, parental leave for a minimum of 12 weeks, bereavement leave, caregiver leave, volunteer leave, well-being reimbursement, programs for financial, physical and mental health. We also offer service anniversary and recognition awards, and subject to the terms of their respective plans, employees - and in some location’s eligible dependents - can participate in several insurance plans. For more information, visit Employee benefits | Supporting well-being & career growth | Johnson & Johnson Careers. (https://www.careers.jnj.com/en/life-at-jj/employee-benefits/)

*This is for informative purposes only. Amounts and actual benefits may vary by location and are subject to change.