Post Doctoral Researcher, Evaluation & Standards – Clinical Operations

Jobtailor; Antwerpen; Sur place; EUR 70 000 - 110 000; Plein temps

Résumé du poste

Jobtailor seeks a researcher to design evaluation frameworks and rubrics for AI agents operating in clinical operations and global development settings. You will define quality dimensions for outputs such as clinical data queries, anomaly detection, and regulatory Q&A content, working with SME s and engineers.

The role involves building benchmark datasets, running human evaluation protocols, and translating findings into measurable improvements for regulated pharma environments.

Qualifications

• PhD or equivalent research experience in a clinical, biomedical, regulatory, biostatistics, or closely related field.
• Strong interest in hands-on work with generative AI and agentic systems.
• Curiosity about LLM behavior in specialized domains and failure modes.
• Proven ability to develop evaluation frameworks, rubrics, and structured assessment protocols.

Responsabilités

• Develop evaluation frameworks and rubrics for AI agents in clinical operations and global development contexts.
• Define quality dimensions for outputs like clinical data queries, anomaly detection, and regulatory documents.
• Design and run human evaluation protocols with clinical, regulatory, and site-facing experts.
• Build benchmark datasets and evaluation pipelines used across AI applications.

Connaissances

Python

ML/Data stack

Analytical writing

Interdisciplinary collaboration

Quality assessment design

Formation

PhD in Clinical/ Biomedical/ Regulatory/ Biostatistics

Outils

LLM APIs

Vector databases

Description du poste

• Develop evaluation frameworks and rubrics for AI agents operating in clinical operations and global development contexts
• Define quality dimensions for specialized outputs, e.g. clinical data queries, anomaly detection, protocol Q&A responses, ICF content, regulatory documents, that reflect how clinical‑ops users actually judge quality
• Design and run human evaluation protocols with clinical, regulatory, and site‑facing subject matter experts
• Build benchmark datasets and evaluation pipelines that the broader GenAI organization uses across AI applications
• Partner with our AI engineering teams to translate evaluation findings into measurable improvement
• Contribute to the methodology of how a regulated pharma organization evaluates AI in clinical work

Requirements

• Strong understanding of clinical operations and global development workflows, e.g. clinical data management, monitoring, regulatory documentation, site operations, or related areas
• Familiarity with how clinical‑ops specialists, monitors, regulatory authors, and site teams actually do their work and assess quality
• PhD or equivalent research experience in a clinical, biomedical, regulatory, biostatistics, or closely related field
• Genuine interest in working hands‑on with generative AI and agentic systems
• Curiosity about how LLMs behave in specialized domains and where they fail
• Comfort prototyping with modern AI tooling (LLM APIs, vector databases, evaluation frameworks)
• Demonstrated ability to develop frameworks, rubrics, and structured assessment protocols
• Experience designing evaluations with multiple quality dimensions, expert raters, or non‑trivial inter‑rater reliability considerations
• Strong analytical and writing skills; comfort with ambiguity and unsolved problems
• Practical skills Python and the modern ML/data stack
• Comfort working across disciplines, e.g. clinical operations, regulatory affairs, AI engineering, data science

Core Competencies

Demonstrates expertise in developing evaluation frameworks and rubrics for AI in clinical operations, with a strong understanding of quality dimensions and workflows in clinical data management and regulatory documentation. Proficient in Python and modern AI tooling, with the ability to collaborate across disciplines to enhance AI applications in regulated environments.

Certifications & Qualifications

• PhD in Clinical, Biomedical, Regulatory, or Biostatistics Field