LLM Application Engineer

Xist4 IT Limited.; City Of London; Remote; GBP 90,000 - 115,000; Full time

Job summary

Xist4 IT Limited is seeking an LLM Application Engineer to own the behaviour, prompts, and evaluation between LLMs and users in a fully remote UK role. You’ll build and ship production-grade Python AI applications, integrating tools, APIs, databases and retrieval systems while managing multi-step workflows.

In this role you’ll own datasets and evaluation frameworks to measure quality, reliability and regressions, with a focus on latency, cost and observability.

Qualifications

• Production Python used to build AI applications.
• Multi-step workflows involving planning, memory or tool calling.
• Evaluation frameworks or datasets you have built for LLM behaviour.
• Retrieval, APIs or external tools integrated into an AI workflow.
• Production AI failures you have investigated across more than one layer.

Responsibilities

• Build LLM-powered applications and multi-step workflows around reasoning, planning, memory and tool use.
• Connect models to APIs, databases, search, internal services and external tools.
• Use prompting, context engineering, structured outputs and tool calling to improve model behavior.
• Debug across prompts, orchestration, backend services and product UX when behavior goes wrong.
• Build datasets and evaluation frameworks that measure quality, reliability and regressions.
• Optimize for quality, latency and cost with observability and tracing built into the system.

Skills

Python

LLM systems

Agent workflows

Evaluation frameworks

Prompts & context

Tool integration

Tools

PyTorch

JAX

Vector databases

Retrieval systems

Open-weight models

Job description

LLM Application Engineer | Python, agents, evaluation | Fully remote, UK

£90,000 to £115,000. Permanent. London. Fully remote across the UK.

The job is to make probabilistic models behave like dependable product features. You will own the prompts, context, tools, orchestration and evaluation that sit between an LLM and a user getting something useful done.

Our client is an early-stage AI product company building applications that get on with everyday tasks before you ask. A prototype exists, launch is ahead, and the company is funded without relying on an upcoming round.

You’ll work across model behaviour, backend services and product experience in a small, distributed technical team. A problem is yours from working out what the user needs through to measuring what the system does in production.

The awkward cases are the work. Agents lose context, tools fail, outputs drift and a prompt change can fix one case while breaking another. You need a way to measure that, not just a good demo.

The job

Agents.

Build LLM-powered applications and multi-step workflows around reasoning, planning, memory and tool use. You'll connect models to APIs, databases, search, internal services and external tools.

Behaviour.

Use prompting, context engineering, structured outputs and tool calling to improve how models act. You’ll debug across prompts, orchestration, backend services and product UX when behaviour goes wrong.

Evaluation.

Build datasets and evaluation frameworks that measure quality, reliability and regressions. Changes need evidence, especially when model behaviour is non-deterministic.

Production.

Optimise for quality, latency and cost, with observability, tracing and experimentation built into the system. The stack includes Python, LLM APIs, open-weight models, vector databases, retrieval systems and PyTorch or JAX.

What you'll bring

Essential:

• AI-powered applications you have shipped to real users.
• LLM or agent systems you have built and run as working software.
• Production Python used to build AI applications.
• Multi-step workflows involving planning, memory or tool calling.
• Evaluation frameworks or datasets you have built for LLM behaviour.
• Retrieval, APIs or external tools integrated into an AI workflow.
• Production AI failures you have investigated across more than one layer.
• Existing right to work in the UK.

Useful: vector databases, RAG, PyTorch, JAX, open-weight models, distributed systems, tracing.

Who this suits

You are probably an LLM Engineer, AI Engineer or Applied AI Engineer who thinks software engineering and evaluation matter as much as model choice. You want to own behaviour after launch, not only get a prototype working.

It won't suit you if you'd rather judge a change by eye. Here every change to model behaviour has to be measured against a dataset you built, and you write the production code around it too.

Let's talk

We welcome applicants from every background and will support reasonable adjustments.