Senior LLM Serving Platform Engineer

Scale AI; Greater London; On-site; GBP 90,000 - 130,000; Full time

Job summary

Scale AI in London, UK seeks an AI Infrastructure Engineer to design and build scalable, reliable serving platforms for LLMs. You will bridge research and engineering to deliver seamless experiences to customers and accelerate innovation across the company.

The role focuses on building fault-tolerant systems, internal discovery platforms for LLMs, and collaborating with researchers to productionize models. 4+ years of backend infra experience is preferred.

Qualifications

• 4+ years of experience building large-scale, high-performance backend systems.
• Proficiency in one or more languages (Python, Go, Rust, C++).
• Experience with LLM serving and routing fundamentals (rate limiting, token streaming, load balancing).
• Experience with LLM capabilities like reasoning, tool calling, prompt templates.

Responsibilities

• Build fault-tolerant, high-performance serving systems for LLMs at scale.
• Develop internal platform for LLM capability discovery.
• Collaborate with researchers to optimize models for production and research.

Skills

Backend systems

Python

Go

Rust

C++

LLM serving

Routing fundamentals

Containers

Kubernetes

Cloud infrastructure

Tools

Docker

Kubernetes

Terraform

Job description

Scale AI in London, UK seeks an AI Infrastructure Engineer to design and build scalable, reliable serving platforms for LLMs. You will bridge research and engineering to deliver seamless experiences to customers and accelerate innovation across the company.

The role focuses on building fault-tolerant systems, internal discovery platforms for LLMs, and collaborating with researchers to productionize models. 4+ years of backend infra experience is preferred.