Senior Life Sciences AI Model Quality & Evaluation Lead

Norstella; United Kingdom; On-site; GBP 70,000 - 95,000; Full time

Benefits offered by this job

25 days annual leave

4 volunteering days + personal day

Group Life Assurance (employer funded)

Group Income Protection (employer)

Job summary

Norstella is seeking a Senior Life Sciences Analyst for Model Training to sit at the intersection of deep clinical domain expertise and applied AI development. You will collaborate with data scientists, ML engineers and data engineers, focusing on model evaluation, calibration, and preference alignment to deliver predictive analytics across clients.

Travel: None required. Applicants must be authorized to work in the UK; Norstella values integrity, empathy, and resilience in a fast-evolving

Qualifications

• Graduate degree in life sciences, medical sciences, computer science or equivalent.
• At least 3 years of professional experience in production-grade life science datasets, including with AI-enabled applications.
• Experience with structured publishing platforms and data tools; comfort with automation concepts
• Experience with large and complex data sets, including data cleaning and preprocessing.
• Excellent problem-solving skills and the ability to work independently.
• Excellent communication skills, especially between technical and non-technical teams.

Responsibilities

• Evaluate model outputs across clinical, regulatory, and life sciences tasks by rating individual responses, ranking pairs and lists, and rewriting outputs to demonstrate the preferred response.
• Define and maintain evaluation rubrics, preference dimensions, and quality criteria (factual accuracy, clinical reasoning quality, etc) that govern model outputs and preference alignment.
• Act as an expert-in-the-loop for how life science models are commercialized with clients, including ongoing support for implementation.
• Advise and assist with interpretation of model behavior, bridging the desired behavior back to the dataflow inflection point, working with the relevant teams to adjust as needed.
• Partner with data scientists and MLEs to walk feedback into data collection pipelines.
• Conduct red-team and adversarial evaluation of models, surfacing subtle failure modes — hallucinations, citation errors, unsupported clinical claims, regulatory missteps - that automated metrics cannot reliably catch.
• Analyze preference data and model behavior to identify systematic gaps, then partner with the fine-tuning SMEs and the data science team to decide how each gap is addressed through preference data.
• Conduct new proofs of concept for novel domain capabilities.
• Contribute to Norstella’s knowledge graph and taxonomy work and help design new agentic workflows based on domain-grounded language models.

Skills

Data analysis

Communication

Independent work

Education

Graduate degree in life sciences/CS

Tools

Structured publishing platforms

Automation concepts

Job description

Norstella is seeking a Senior Life Sciences Analyst for Model Training to sit at the intersection of deep clinical domain expertise and applied AI development. You will collaborate with data scientists, ML engineers and data engineers, focusing on model evaluation, calibration, and preference alignment to deliver predictive analytics across clients.

Travel: None required. Applicants must be authorized to work in the UK; Norstella values integrity, empathy, and resilience in a fast-evolving