Director, Search & AI Evaluation

RELX; Greater London; On-site; GBP 140,000 - 210,000; Full time

Benefits offered by this job

Comprehensive Pension Plan

Flexible working hours

Sabbatical leave

Employee discounts

Parental leave

Job summary

Elsevier is seeking a Director of Data Science – Search & Evaluation to lead strategy, vision, and organizational growth for AI-enabled discovery. You will manage Senior Data Scientists and Analysts, shaping evaluation frameworks, search relevance, and AI quality across platforms.

You will partner with Product, Engineering, and UX Research to translate complex findings into strategic guidance, drive governance, and mentor a high-performing team in a fast-moving AI environment.

Qualifications

• Master’s or PhD in Computer Science, Data Science, ML, IR, Statistics, NLP, or related field.
• Significant experience in Data Science, ML, IR, search relevance, evaluation systems, or applied AI.
• Experience leading and scaling high-performing technical teams in complex, cross-functional orgs.
• Deep expertise in search relevance, information retrieval, and evaluation methodologies.

Responsibilities

• Define and drive long-term strategy for search relevance, retrieval evaluation, ranking optimization, and AI system quality.
• Lead initiatives to improve search relevance, semantic retrieval, RAG, grounding, and user discovery outcomes.
• Establish scalable evaluation methodologies for discovery, recommendation, and LLM-powered systems.
• Guide experimentation across lexical, semantic, hybrid, and AI-assisted retrieval architectures.
• Partner with Product and Engineering to align investments with business priorities.

Skills

Search relevance

Information retrieval

Experimentation

AI evaluation

Leadership

Python

Communication

Education

Master’s or PhD in CS/DS/ML/IR/NLP

Tools

PyTorch

Hugging Face

LangChain

LangGraph

Haystack

Job description

Search & AI evaluation

Ready to lead a data science organisation that pushes the boundaries of what intelligent systems can achieve?
Do you thrive on shaping strategy, inspiring teams, and delivering solutions that create real‑world impact?

Director, Search & AI evaluation

Ready to lead a data science organisation that pushes the boundaries of what intelligent systems can achieve?
Do you thrive on shaping strategy, inspiring teams, and delivering solutions that create real‑world impact?

About The Team

Elsevier’s mission is to help researchers, clinicians, and life sciences professionals advance discovery and improve health outcomes through trusted content, data, and analytics. Our Search & Evaluation organization plays a critical role in shaping the future of AI-powered discovery by building intelligent retrieval, ranking, and evaluation systems that power trusted scientific and healthcare experiences.
The team is responsible for advancing search relevance, retrieval quality, experimentation frameworks, and AI evaluation capabilities across Elsevier’s next-generation AI platforms and products.
We combine expertise in information retrieval, machine learning, analytics, experimentation, and generative AI to improve how users discover, synthesize, and interact with scientific knowledge.
Our work spans semantic search, retrieval-augmented generation (RAG), ranking systems, LLM evaluation, online experimentation, and scalable evaluation infrastructure. We partner closely with Product, Engineering, UX Research, Knowledge Graph, and Applied AI teams to deliver measurable improvements in AI quality and user outcomes.

About The Role

We are looking for a Director, Data Science – Search & Evaluation, to lead the strategic direction, technical vision, and organizational development of our Search & Evaluation function.
This role will focus on defining and scaling evaluation frameworks, search relevance methodologies, retrieval optimization strategies, and AI quality measurement systems across Elsevier’s AI-powered discovery experiences.
You will lead a multidisciplinary team of Senior Data Scientists and Data Analysts responsible for evaluating and improving search, ranking, retrieval, recommendation, and generative AI systems. You will help establish best practices for experimentation, evaluation rigor, and AI quality while influencing product strategy and technical direction across multiple initiatives.
This role is ideal for a leader with deep expertise in search relevance, information retrieval, experimentation, and AI evaluation who can combine technical depth, strategic thinking, organizational leadership, and strong cross-functional influence.

Key Responsibilities

Search, Retrieval & AI Quality Strategy

• Define and drive the long-term strategy for search relevance, retrieval evaluation, ranking optimization, and AI system quality.
• Lead initiatives focused on improving:
  • Search relevance and ranking quality.
  • Semantic retrieval and vector search
  • Retrieval-augmented generation (RAG)
  • AI grounding and hallucination mitigation
  • User discovery and engagement outcomes
• Establish scalable evaluation methodologies for search, retrieval, recommendation, and LLM-powered systems.
• Guide experimentation and optimization across lexical, semantic, hybrid, and AI-assisted retrieval architectures.
• Partner with Product and Engineering leadership to align search and AI investments with customer and business priorities.
• Influence technical direction for retrieval systems, evaluation infrastructure, and AI quality frameworks across platforms.

Evaluation & Experimentation Leadership

• Define and operationalize evaluation frameworks for search and generative AI systems, including:
  • IR metrics (e.g., NDCG, recall, precision)
  • LLM and RAG evaluation methodologies
  • Grounding and faithfulness evaluation
  • Human evaluation and annotation strategies
  • Online experimentation and A/B testing
• Establish best practices for offline benchmarking, online experimentation, and reproducible evaluation workflows.
• Build scalable processes for benchmark creation, annotation quality, evaluation governance, and performance reporting.
• Drive rigorous, evidence-based decision-making across AI and search initiatives.
• Champion responsible AI practices focused on quality, reliability, trust, and measurable user impact.

Organizational & Cross-functional Leadership

• Lead, mentor, and grow a high-performing team of Data Scientists and Analysts.
• Create a culture of scientific rigor, accountability, collaboration, innovation, and continuous learning.
• Partner closely with Product Managers, Engineers, UX Researchers, and Applied AI teams to deliver impactful AI capabilities.
• Translate complex technical findings into clear business insights and strategic recommendations for senior stakeholders and executive leadership.
• Help define organizational priorities, roadmaps, and operating models for Search & Evaluation initiatives.
• Drive alignment across cross-functional teams operating in fast-moving and ambiguous AI environments.
• Contribute to long-term AI and discovery strategy across Elsevier platforms.

Requirements

• Master’s or PhD in Computer Science, Data Science, Machine Learning, Information Retrieval, Statistics, NLP, or a related quantitative field.
• Significant experience in Data Science, Machine Learning, Information Retrieval, Search Relevance, Evaluation Systems, or Applied AI.
• Significant experience leading and scaling high-performing technical teams in complex, cross-functional organizations.
• Deep expertise in:
  • Search relevance and ranking systems.
  • Information retrieval and semantic search
  • Retrieval-augmented generation (RAG)
  • Evaluation methodologies for IR and generative AI systems
  • Experimentation frameworks and A/B testing
• Strong experience with:
  • Vector retrieval and hybrid search architectures.
  • LLM evaluation and AI quality measurement
  • Embeddings, reranking, and retrieval orchestration
  • Evaluation datasets, benchmarking, and annotation workflows
• Advanced programming skills in Python.
• Experience with modern AI/ML frameworks and tooling (e.g., PyTorch, Hugging Face, LangChain, LangGraph, Haystack).
• Experience working with large-scale datasets, distributed data/ML platforms, and production AI systems.
• Strong understanding of statistical analysis, experimentation design, and evaluation science.
• Excellent communication and stakeholder management skills, including experience influencing senior leadership.
• Demonstrated ability to balance strategic leadership with pragmatic execution in rapidly evolving AI environments.

Preferred Qualifications

• PhD preferred in Computer Science, Machine Learning, NLP, Information Retrieval, Statistics, or related field.
• Experience leading search, ranking, recommendation, or AI evaluation organizations at scale.
• Experience building evaluation systems for LLM-powered applications and AI assistants.
• Familiarity with scientific, biomedical, or scholarly datasets and workflows.
• Experience with knowledge graphs, ontologies, or semantic enrichment systems.
• Exposure to production ML systems, MLOps, and AI governance practices.
• Publications or applied research contributions in NLP, IR, search, recommendation systems, or generative AI.
• Experience building AI systems in high-trust, regulated, or content-rich domains.

Why join us?

Join our team and contribute to a culture of innovation, collaboration, and excellence.

Work in a way that works for you

We promote a healthy work/life balance across the organization. We offer an appealing working prospect for our people. With numerous wellbeing initiatives, shared parental leave, study assistance, and sabbaticals, we will help you meet your immediate responsibilities and your long‑term goals.

• Working flexible hours - flexing the times when you work in the day to help you fit everything in and work when you are the most productive.

Working for you

Benefits

We know that your well‑being and happiness are key to a long and successful career. These are some of the benefits we are delighted to offer:

• Comprehensive Pension Plan
• Home, office, or commuting allowance.
• Generous vacation entitlement and option for sabbatical leave
• Maternity, Paternity, Adoption, and Family Care leave
• Flexible working hours
• Personal Choice budget
• Internal communities and networks
• Various employee discounts
• Recruitment introduction reward
• Employee Assistance Program (global)

About The Business –

A global leader in information and analytics, we help researchers and healthcare professionals advance science and improve health outcomes for the benefit of society. Building on our publishing heritage, we combine quality information and vast data sets with analytics to support visionary science and research, health education, and interactive learning, as well as exceptional healthcare and clinical practice. At Elsevier, your work contributes to the world’s grand challenges and a more sustainable future. We harness innovative technologies to support science and healthcare to partner for a better world.