AI Infrastructure Engineer - Scalable LLM Serving

Mat Vin; Greater London; Hybrid; GBP 90,000 - 150,000; Full time

Job summary

Mat Vin is seeking an AI Infrastructure Engineer to design and build scalable platforms for serving LLMs. You will join a collaborative ML Infrastructure team, bridging research and engineering to deliver reliable systems used across internal and external customers in London.

You will own fault-tolerant backend design, containerized deployments, and observability, working with researchers and engineers to optimize production and research use cases.

Qualifications

• 4+ years building large-scale backend systems.
• Proficient in Python, Go, Rust, or C++.
• Experience with LLM serving and routing fundamentals.

Responsibilities

• Build and maintain fault-tolerant, high-performance systems for serving LLMs and other models at scale.
• Build an internal platform to empower LLM capability discovery.
• Collaborate with researchers and engineers to integrate and optimize models for production and research use cases.
• Conduct architecture and design reviews to uphold best practices in system design and scalability.
• Develop monitoring and observability solutions to ensure system health and performance.
• Lead projects end-to-end, from requirements gathering to implementation, in a cross-functional environment.

Skills

Backend systems

Python

Go

Rust

C++

LLM serving

System design

Monitoring

Independent work

Tools

Docker

Kubernetes

Terraform

AWS

GCP

Job description

Mat Vin is seeking an AI Infrastructure Engineer to design and build scalable platforms for serving LLMs. You will join a collaborative ML Infrastructure team, bridging research and engineering to deliver reliable systems used across internal and external customers in London.

You will own fault-tolerant backend design, containerized deployments, and observability, working with researchers and engineers to optimize production and research use cases.