LLM Applications Engineer — Build & Evaluate AI Workflows

Xist4 IT Limited.; City Of London; Remote; GBP 90,000 - 115,000; Full time

Job summary

Xist4 IT Limited is seeking an LLM Application Engineer to own the behaviour, prompts, and evaluation between LLMs and users in a fully remote UK role. You’ll build and ship production-grade Python AI applications, integrating tools, APIs, databases and retrieval systems while managing multi-step workflows.

In this role you’ll own datasets and evaluation frameworks to measure quality, reliability and regressions, with a focus on latency, cost and observability.

Qualifications

• Production Python used to build AI applications.
• Multi-step workflows involving planning, memory or tool calling.
• Evaluation frameworks or datasets you have built for LLM behaviour.
• Retrieval, APIs or external tools integrated into an AI workflow.
• Production AI failures you have investigated across more than one layer.

Responsibilities

• Build LLM-powered applications and multi-step workflows around reasoning, planning, memory and tool use.
• Connect models to APIs, databases, search, internal services and external tools.
• Use prompting, context engineering, structured outputs and tool calling to improve model behavior.
• Debug across prompts, orchestration, backend services and product UX when behavior goes wrong.
• Build datasets and evaluation frameworks that measure quality, reliability and regressions.
• Optimize for quality, latency and cost with observability and tracing built into the system.

Skills

Python

LLM systems

Agent workflows

Evaluation frameworks

Prompts & context

Tool integration

Tools

PyTorch

JAX

Vector databases

Retrieval systems

Open-weight models

Job description

Xist4 IT Limited is seeking an LLM Application Engineer to own the behaviour, prompts, and evaluation between LLMs and users in a fully remote UK role. You’ll build and ship production-grade Python AI applications, integrating tools, APIs, databases and retrieval systems while managing multi-step workflows.

In this role you’ll own datasets and evaluation frameworks to measure quality, reliability and regressions, with a focus on latency, cost and observability.