Senior ML Infrastructure Engineer

Ellison Institute of Technology; Oxford; On-site; GBP 90,000 - 130,000; Full time

Benefits offered by this job

Travel allowance

Pension 7.5%

Private Medical Insurance

Life Assurance

Cycle to Work

Job summary

Ellison Institute of Technology (EIT) invites applications for a role leading the design, build, and operation of high‑performance ML compute clusters at scale in Oxford. The role focuses on reliable, secure platforms, automated guardrails, and scalable experimentation pipelines.

You will collaborate with Research, Data, and Applied teams to forecast capacity and drive cost efficiency. You must be onsite in Oxford with a willingness to travel; sponsorship possibilities may be assessed

Qualifications

• Proven experience leading the design, build, and operation of high‑performance ML compute clusters at scale.
• Autonomous systems design with ability to ideate and implement optimal solutions.
• Exposure to migrating ML infrastructure to containerised systems.
• Expertise with high‑throughput storage systems for ML/HPC workloads.
• Expert-level understanding of GPU architecture and high‑speed networking for distributed training.

Responsibilities

• Build, operate, and optimise high‑performance GPU training/inference clusters for scalable ML workloads.
• Design and implement high‑throughput data paths, optimise I/O and data locality across compute/storage.
• Benchmark, profile, and resolve performance bottlenecks across compute, network and orchestration layers.
• Establish observability, resilience, and automated security controls for sensitive research environments.
• Forecast capacity and cost for GPU/storage needs with cross‑team collaboration.

Skills

ML compute clusters

GPU architectures

High-throughput storage

Iac and CI/CD

Terraform

Argo CD

Tools

Terraform

Argo CD

Job description

Join us at EIT: At the Ellison Institute of Technology (EIT), we're on a mission to translate scientific discovery into real world impact. We bring together visionary scientists, technologists, policy makers, and entrepreneurs to tackle humanity's greatest challenges in four transformative areas: Health, Medical Science & Generative Biology Food Security & Sustainable Agriculture Climate Change & Managing CO₂ Artificial Intelligence & Robotics This is ambitious work - work that demands curiosity, courage, and a relentless drive to make a difference. At EIT, you'll join a community built on excellence, innovation, tenacity, trust, and collaboration, where bold ideas become real-world breakthroughs. Together, we push boundaries, embrace complexity, and create solutions to scale ideas for lab to society. Explore more at www.eit.org

Your Role:

Join our SciComp team to build the cloud and compute foundation that enables scientific breakthroughs. Deliver reliable, secure platforms and self-service guardrails that accelerate experimentation and turn ideas into results - faster, at scale, and with confidence.

Your Responsibilities:

• Build, operate, and continuously optimise our high-performance GPU training and inference clusters, focusing on robust, high-availability scheduling, isolation, and automated lifecycle management.
• Drive systems design and implementation for high-throughput data paths, optimising I/O, caching, and data locality across compute and storage (including our current Lustre implementation).
• Proactively benchmark, profile, and resolve performance bottlenecks across the compute, network, and orchestration layers to maximise efficiency for distributed training and inference.
• Establish comprehensive observability, resilience, and automated security controls to ensure compliance and robust operation of sensitive research environments.
• Partner with Research, Data, and Applied teams to forecast capacity and cost for GPU and storage needs, setting quotas and streamlining ML experimentation pipelines.

Essential Skills, Qualifications & Experience:

• Proven experience leading the design, build, and operation of high-performance ML compute clusters at scale
• A proactive, autonomous approach to systems design and the proven ability and desire to ideate, co-create and implement optimal solutions
• Exposure to migrating or transforming ML infrastructure from traditional schedulers to modern, containerised systems
• Expertise with high-throughput storage systems for ML/HPC workloads
• Expert-level understanding of GPU architecture, high-speed networking for distributed training, and performance profiling to resolve bottlenecks
• A solid grasp of IaC and CI/CD practices (e.g., Terraform, Argo CD)

We offer the following salary and benefits:

• Competitive salary (dependent on experience) + travel allowance + bonus
• Enhanced holiday. Our annual leave allowance is 25 days plus 8 bank holidays and an additional 3 days between Christmas and New Year. You will also have the opportunity to purchase an additional 5 days annual leave in January and July.
• Pension - Employer contribution 7.5%, minimum employee contribution 5%
• Life Assurance.
• Income Protection
• Private Medical Insurance as standard for you, your partner and any dependents. Including hospital Cash Plan
• Employee discounts
• Electric car scheme
• Nursery Salary Sacrifice scheme
• Cycle to Work Scheme
• Family Planning
• Neurodiversity support including advise and assessments
• Coaching & Therapy services

Working together - what it involves: You must have the right to work permanently in the UK with a willingness to travel as necessary. In certain cases, we can consider sponsorship, and this will be assessed on a case-by-case basis. You will live in, or within easy commuting distance of, Oxford (or be willing to relocate) and can commit to being onsite at our Oxford office, a minimum of 3 days per working week.