AI Infra Engineer: Scalable LLM Serving & Platform Design

scaleai; Greater London; On-site; GBP 110,000 - 160,000; Full time

Job summary

Scale AI is seeking a Software Engineer to design and build scalable ML infrastructure for serving LLMs. You will work across research and engineering to deliver reliable, high-performance platforms that power both internal and external use cases.

You will collaborate with researchers and engineers to optimize models for production and research, and you will lead end-to-end initiatives in a fast-moving environment. A strong backend and ML grounding are essential for success.

Qualifications

• Strong ML fundamentals and backend system design experience.
• 4+ years building large-scale backend systems.
• Experience with LLM serving and routing fundamentals.

Responsibilities

• Build and maintain fault-tolerant, high-performance systems for serving LLMs and other models at scale.
• Build an internal platform to empower LLM capability discovery.
• Collaborate with researchers and engineers to integrate and optimize models for production and research use cases.
• Conduct architecture and design reviews to uphold best practices in system design and scalability.
• Develop monitoring and observability solutions to ensure system health and performance.
• Lead projects end-to-end, from requirements gathering to implementation, in a cross-functional environment.

Skills

Python

Go

Rust

C++

LLM serving

System design

Tools

Docker

Kubernetes

Terraform

AWS

GCP

Job description

Scale AI is seeking a Software Engineer to design and build scalable ML infrastructure for serving LLMs. You will work across research and engineering to deliver reliable, high-performance platforms that power both internal and external use cases.

You will collaborate with researchers and engineers to optimize models for production and research, and you will lead end-to-end initiatives in a fast-moving environment. A strong backend and ML grounding are essential for success.