Remote RLHF Program Manager: AI Alignment

Coaley Peak; Cheltenham; Hybrid; GBP 55,000 - 75,000; Full time

Benefits offered by this job

28 days annual leave

Private healthcare

Professional development budget £2,000

Job summary

Coaley Peak is seeking an RLHF Manager to lead reinforcement learning from human feedback across its internal AI engines and external client projects. You will design reward models, manage human evaluator programmes, and evaluate model alignment against business objectives and UK regulatory expectations.

The role is dual-facing, with responsibilities for internal RLHF alignment and external client work in regulated sectors.

Qualifications

• Deep understanding of RLHF, preference learning, and alignment concepts applicable to business goals.
• Experience managing structured data annotation, human evaluation, or labelling programmes.
• Proficiency in Python; familiarity with ML frameworks and LLM APIs.

Responsibilities

• Lead RLHF programmes across internal AI engines and client projects; define reward models.
• Manage annotation and evaluation pipelines; scope requirements and delivery.
• Measure output quality over time and report findings to client technical steering groups.
• Coordinate with data scientists, engineers, and non-technical stakeholders.

Skills

RLHF concepts

Annotation management

Python proficiency

Project management

Technical writing

Communication with stakeholders

Tools

ML frameworks

LLM APIs (OpenAI, Anthropic, HF)

Job description

Coaley Peak is seeking an RLHF Manager to lead reinforcement learning from human feedback across its internal AI engines and external client projects. You will design reward models, manage human evaluator programmes, and evaluate model alignment against business objectives and UK regulatory expectations.

The role is dual-facing, with responsibilities for internal RLHF alignment and external client work in regulated sectors.