ML Scheduler Engineer: Kubernetes & GPU/TPU Orchestration

Isomorphic Labs; City of Westminster; On-site; GBP 90,000 - 150,000; Full time

Job summary

Isomorphic Labs is seeking an engineer to own ML workload scheduling on Kubernetes, optimizing dynamic batch inference and large-scale training runs. You will design scalable scheduling systems, write custom controllers, and collaborate with ML researchers to maximize hardware utilization and accelerate AI-driven drug discovery.

The role requires deep Kubernetes expertise, strong Python or Go skills, and hands-on experience with cloud infrastructure and HPC-style workloads.

Qualifications

• Deep Kubernetes expertise and hands-on experience
• Strong programming in Python or Go
• Experience with cloud providers and IaC
• Familiarity with ML infrastructure concepts
• Background in HPC or GPU/TPU utilization
• Knowledge of cluster schedulers and batch systems

Responsibilities

• Own the Scheduling Layer: Design, operate, and extend ML workload scheduling systems for dynamic batch inference and long-running training workloads.
• Optimize Compute Utilization: Implement scheduling policies for efficient use across GPUs/TPUs.
• Partner with Research: Collaborate with ML researchers and Infrastructure to translate workload requirements into platform capabilities.
• Ensure Reliability: Build systems with clear failure semantics and maintain high availability with SLOs.
• Build Custom Infrastructure: Develop Kubernetes controllers, operators, and CRDs for ML pipelines.
• Scheduling Frameworks: Work with cluster schedulers or queuing frameworks like Kueue, Volcano, Slurm.
• Deep Kubernetes Expertise: Write and maintain custom K8s controllers and operators and interact with API.
• Cloud Infrastructure Experience: Provisioning, networking, and IaC for scalable environments.
• Adaptability: Apply DevOps/SRE best practices to AI accelerator infrastructure.
• Familiarity with ML/AI hardware: Nvidia GPUs, Google TPUs, topology-aware scheduling.
• Background in ML SWE or infra SRE supporting ML research.

Skills

Kubernetes

Python

Go

Distributed systems

Cloud (GCP)

SRE/DevOps

Infrastructure as Code

HPC

GPU/TPU orchestration

ML infra

Job description

Isomorphic Labs is seeking an engineer to own ML workload scheduling on Kubernetes, optimizing dynamic batch inference and large-scale training runs. You will design scalable scheduling systems, write custom controllers, and collaborate with ML researchers to maximize hardware utilization and accelerate AI-driven drug discovery.

The role requires deep Kubernetes expertise, strong Python or Go skills, and hands-on experience with cloud infrastructure and HPC-style workloads.