AI Lead

Marker

Greater London

Hybrid

GBP 120,000 - 150,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Calm work environment
Creative work opportunities
Growth opportunities
Direct collaboration with leadership

Job summary

An innovative AI company located in London seeks an experienced AI practitioner to lead evaluation efforts for AI-driven writing tools. The ideal candidate will collaborate closely with leadership to enhance systems that assist writers, utilizing their expertise in AI/ML methodologies and evaluation frameworks. This role offers a creative environment with a focus on the intersection of AI and human creativity.

Qualifications

  • Experience with AI/ML evaluation methodologies and can speak the language of AI research.
  • Hands-on experience with language models and understanding of evaluating subjective outputs.
  • Self-starter willing to work independently.

Responsibilities

  • Lead AI endeavours to enhance writing assistance.
  • Implement evaluation frameworks for complex AI outputs.
  • Collaborate with engineering to integrate evaluation insights.

Skills

AI/ML evaluation methodologies
Experience with language models
Critical thinking about metrics
Python programming
Understanding of subjective outputs

Education

Postgraduate or equivalent experience in AI or related field

Tools

Python
TypeScript

Job description

AI Evaluation, Research Methods, Python, LLMObservability

Salary range

£120,000-£150,000 p.a. + equity, depending on experience (up to £180,000 forcandidates with exceptional relevant experience)

Apply

Email us at work@writewithmarker.com and tell us a little bit about yourselfand your interest in the future of writing, along with your CV or a link to your CV site.

What is Marker?

Marker is an AI-native Word Processor – a reimagining of Google Docs and Microsoft Word.

Join us in building the next generation of agentic AI assistants supporting serious writers in their work.

We are a small, ambitious company using cutting-edge technology to give everybody writing superpowers.

What you'll do at Marker

We are looking for a skilled AI practitioner and thinker – someone with experience in academia and/or industry with a drive to help writers and push forward the state-of-the-art in AI in creative fields.Below is our first draft of your role but we imagine working with you to co-create your precise role.

You will lead our AI endeavours and help us bring rigour and insight to our AI systems through invention, evaluation, research, and observability. You'll work directly with Ryan Bowman (CPO) and Jon Steinback (CEO) to help us understand and improve how our AI assists writers.Here are some examples of areas you will be working in:

  • Design and implement evaluation frameworks for complex, subjective AI outputs (like writing feedbackthat's meant to inspire rather than just correct)
  • Build flexible evaluation pipelines that can assess quality across multiple dimensions - from humanpreference to actual writing improvement
  • Work together with our engineering team to new, fun and useful AI features that help writers
  • Use data from our platform to discover novel integrations of AI in writing and publish those findings
  • Research and prototype new evaluation methodologies for creative and subjective AI tasks
  • Collaborate with our engineering team to integrate evaluation insights into our development process
  • Help define what "quality" means for different AI outputs and create metrics that actually matter forour users
  • Work on challenging problems like: "How do we automatically evaluate whether an AI comment successfullyencourages thoughtful revision?"
What we can offer
  • A calm, human-friendly work environment among kind and experienced professionals
  • Fun, creative, novel, and interesting technical work at the intersection of AI research and productdevelopment
  • An opportunity to work with and learn about the latest advancements in AI evaluation and language models
  • Direct collaboration with leadership to shape how we understand and improve our AI systems
  • As much responsibility and growth opportunities as you want to take on
Are you a good fit for this role?

In order to be successful in this role, you will recognise yourself in the following:

  • You have experience with AI/ML evaluation methodologies and can speak the language of AI research
  • You've worked hands-on with language models and understand the challenges of evaluating subjective,creative outputs
  • You are a self-starter willing to work independently and at speed
  • You are interested in helping writers expand their craft and hone their work
  • You have published work or dissertation level experience with related subjects
  • You are familiar with and have worked on related technical systems (evaluation pipelines, datacollection tools) but don't need to be a full-stack engineer. You won't be expected to build these alone!
  • You think critically about what metrics actually matter and aren't satisfied with vanity metrics
  • You're comfortable working with ambiguous problems where the "right answer" isn't obvious
  • You have some programming experience (Python preferred) and can work independently on technical projects
  • You're interested in the intersection of AI capabilities and human creativity

An exceptional candidate for this role would be able to demonstrate some of thefollowing:

  • Experience building evaluation systems for generative AI in production environments
  • Knowledge of TypeScript and ability to integrate with our existing systems
  • Background in human-computer interaction, computational creativity, or writing research
  • Experience with A/B testing, statistical analysis, and experimental design
  • Familiarity with modern AI observability and monitoring tools
  • Published research or deep interest in AI evaluation methodologies
  • Interest in writing (fiction, non-fiction, essays)

However, you are NOT expected to:

  • Be a senior software engineer - we're looking for someone who can build evaluation systems, notarchitect our entire backend
  • Have solved every evaluation problem before - this is cutting-edge work and we're figuring it outtogether
  • Be experienced with every library in our stack from day one - you'll work closely with Ryan and ourengineering team
Our stack

You'll be working with the following technologies:

  • Our AI engine uses a range of models, including self-hosted and fine-tuned open source models, as wellas latest reasoning models from Anthropic and OpenAI
  • Evaluation and research tools built primarily in Python, with integration into our TypeScriptinfrastructure
  • Our agentic AI execution platform is written in TypeScript, hosted on Cloudflare Workers
  • Standard ML tooling: various evaluation frameworks, data analysis tools, and monitoring systems
  • Our text editor frontend is a web application built with React, TypeScript and ProseMirror
Apply now!

Interested? Email us at work@writewithmarker.com with your CV (or a link to your CV site).Tell us a little bit about yourself and why you'd like to work at Marker!

Please note that this role is currently only available based in ourLondon hub, and at this time we are not able to sponsor work visas in the UK.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Product Designer — Design Systems for Writers
Product Designer — Design Systems for Writers

Marker • Greater London

On-site
GBP 60,000 - 80,000
Calm work environment
Opportunity for creative work
Collaboration with leadership
+1
AI engineer
AI engineer

Writer • Greater London

Hybrid
GBP 120,000 - 180,000
Health insurance
Parental leave
Hybrid work
+2
Member of Technical Staff, Evaluations
Member of Technical Staff, Evaluations

Ai • Greater London

On-site
GBP 85,000 - 130,000
30 days annual leave
Pension plan with 10% employer contrib
Bupa private health cover
+1
Software engineer, generative AI (UK)
Software engineer, generative AI (UK)

AItoolnavio • Greater London

Hybrid
GBP 90,000 - 130,000
Generous PTO
Medical and dental insurance
Parental leave (16 weeks)
+3
Software Engineer - AI Evaluation
Software Engineer - AI Evaluation

Board Intelligence • Greater London

Hybrid
GBP 70,000 - 100,000
Pension scheme
Personal performance bonus
26 days holiday each calendar year
+4
Senior Frontend Developer (AI Writing Platform)
Senior Frontend Developer (AI Writing Platform)

Jobspage • Greater London

Remote
GBP 51,272 - 102,545
Member of Technical Staff, Evaluations
Member of Technical Staff, Evaluations

Stealth - AI x Bio • Greater London

On-site
GBP 90,000 - 150,000
Senior solutions architect (UK)
Senior solutions architect (UK)

FDE Times • Greater London

Hybrid
GBP 100,000 - 150,000
Generous PTO
Medical and dental insurance
Parental leave (16 weeks)
+6
Senior solutions architect (UK)
Senior solutions architect (UK)

WRITER • Greater London

Hybrid
GBP 110,000 - 150,000
Generous PTO
Company holidays
Medical & dental insurance
+7
Technical Lead – AI & Software Engineering - 7720
Technical Lead – AI & Software Engineering - 7720

Cambridge University Press Ltd. • Cambridge

Hybrid
GBP 53,000 - 71,000
28 days annual leave
Private medical insurance
Discretionary annual bonus
+3