AI Infra Engineer for Scalable LLM Serving

Scale AI, Inc.; Greater London; On-site; GBP 110,000 - 160,000; Full time

Job summary

Scale AI, Inc. in London seeks an AI Infrastructure Engineer to design and build scalable serving platforms for LLMs. You will bridge research and production, delivering high-performance, fault-tolerant backend systems.

Ideal candidates have 4+ years building large-scale services, strong Python/Go/Rust/C++, and experience with Docker/Kubernetes, AWS/GCP, Terraform, and LLM routing concepts.

Qualifications

• 4+ years of experience building large-scale, high-performance backend systems.

Responsibilities

• Build and maintain fault-tolerant, high-performance systems for serving LLMs and other models at scale.
• Build an internal platform to empower LLM capability discovery.
• Collaborate with researchers and engineers to integrate and optimize models for production and research use cases.
• Conduct architecture and design reviews to uphold best practices in system design and scalability.
• Develop monitoring and observability solutions to ensure system health and performance.
• Lead projects end-to-end, from requirements gathering to implementation, in a cross-functional environment.

Skills

Backend systems

Python

Go

Rust

C++

LLM serving

Load balancing

Rate limiting

Containers

Kubernetes

Docker

AWS

GCP

Terraform

vLLM

SGLang

TensorRT-LLM

text-generation-inference

Tools

Python

Go

Rust

C++

Job description

Scale AI, Inc. in London seeks an AI Infrastructure Engineer to design and build scalable serving platforms for LLMs. You will bridge research and production, delivering high-performance, fault-tolerant backend systems.

Ideal candidates have 4+ years building large-scale services, strong Python/Go/Rust/C++, and experience with Docker/Kubernetes, AWS/GCP, Terraform, and LLM routing concepts.