Senior ML Engineer

Arrows; Greater London; On-site; GBP 100,000 - 150,000; Full time

Job summary

Arrows is seeking a lead ML engineer to shape and own the agent harness that plans, executes, and refines generation work using tools, context, memory and long-running runs.

You will collaborate with the CTO and backend teams, set how experiments are run, and drive measurable improvements through data-driven evaluation and iteration.

Qualifications

• BSc, MSc or PhD in Computer Science, Mathematics, Statistics, Physics or a related field, or equivalent experience. Strong Python.
• 4+ years in ML engineering or data science, including production systems you owned end to end.
• You’ve built and shipped an agent system that real users or workloads depend on: models calling tools, planning across steps and acting on the results. A chat interface over retrieval alone is not enough.
• A strong foundation in statistics, experimental design and research, with the ability to separate genuine improvement from noise.
• Experience building evaluation systems for ML or LLM behaviour, turning ambiguous quality questions into metrics and decisions.
• Strong understanding of modern LLM techniques: context engineering, structured outputs, tool use and fine tuning.
• You follow academic research closely and have explored novel techniques or architectures for agents.
• Comfortable across the stack, from model level experiments to deployed services and data pipelines.
• You’ve led technical work: set direction, reviewed designs and mentored more junior colleagues.
• Systems or projects you can discuss in technical depth.

Responsibilities

• Design and build the harness end to end: planning, tool use, context, memory and multistep execution across long running generation work.
• Own the evals: datasets, offline replays of real runs, scorers and model judges, and regression alerts that show when the harness gets better or worse.
• Run experiments and A/B tests on prompts, models, tools and agent design, and approach the results with statistical rigour.
• Close the loop between audience response and agent behaviour, turning performance data into the signals, features and training data the harness learns from.
• Choose and adapt models for each step: frontier APIs, open weight models, and fine tuning where it beats prompting on quality, cost or latency.
• Follow the research closely and bring new agent techniques and architectures into the harness.
• Trace failure modes across the full agent loop, from planning to tool calls to final output.
• Set the standard for ML engineering: experiment design, reproducibility, documentation, and guidance for other engineers.

Skills

Python programming

ML engineering

Experiment design

Statistics basics

LLM techniques

Education

BSc/MSc/PhD in CS or related

Job description

My client is an early stage AI startup founded out of a leading university machine learning lab. Having bootstrapped to a team of 10, they have just closed a £4 million pre seed round and are now on the next leg of growth.

AI can generate almost any form of media, but it still can’t judge what captures human attention. As AI generated content grows and creative workflows become agent assisted or fully agentic, my client is focused on improving how models reason about creative decisions by learning from how real audiences respond.

The role

You’ll lead engineering on their agent harness: the system that plans, generates and refines output, and learns from how real audiences respond to it.

You’ll have the autonomy to set how the team builds, measures and improves its agents, with scope to grow into technical leadership. You’ll work closely with the CTO on direction, with backend engineers on the infrastructure the harness runs on, and with the engineers who use it every day.

What you’ll do

• Design and build the harness end to end: planning, tool use, context, memory and multistep execution across long running generation work.
• Own the evals: datasets, offline replays of real runs, scorers and model judges, and regression alerts that show when the harness gets better or worse.
• Run experiments and A/B tests on prompts, models, tools and agent design, and approach the results with statistical rigour.
• Close the loop between audience response and agent behaviour, turning performance data into the signals, features and training data the harness learns from.
• Choose and adapt models for each step: frontier APIs, open weight models, and fine tuning where it beats prompting on quality, cost or latency.
• Follow the research closely and bring new agent techniques and architectures into the harness.
• Trace failure modes across the full agent loop, from planning to tool calls to final output.
• Set the standard for ML engineering: experiment design, reproducibility, documentation, and guidance for other engineers.

What my client is looking for

• BSc, MSc or PhD in Computer Science, Mathematics, Statistics, Physics or a related field, or equivalent experience. Strong Python.
• 4+ years in ML engineering or data science, including production systems you owned end to end.
• You’ve built and shipped an agent system that real users or workloads depend on: models calling tools, planning across steps and acting on the results. A chat interface over retrieval alone is not enough.
• A strong foundation in statistics, experimental design and research, with the ability to separate genuine improvement from noise.
• Experience building evaluation systems for ML or LLM behaviour, turning ambiguous quality questions into metrics and decisions.
• Strong understanding of modern LLM techniques: context engineering, structured outputs, tool use and fine tuning.
• You follow academic research closely and have explored novel techniques or architectures for agents.
• Comfortable across the stack, from model level experiments to deployed services and data pipelines.
• You’ve led technical work: set direction, reviewed designs and mentored more junior colleagues.
• Systems or projects you can discuss in technical depth.

Nice to have

• Post training experience: SFT, RLVR, RLHF, preference optimisation or RL for agent behaviour.
• Ranking, recommendation or causal inference work, especially predicting engagement.
• Multimodal models: image, video or audio generation and understanding.
• Founding or early engineer experience at a startup, or equivalent ownership inside a larger company.