Senior AI Inference Engineer - Open-Source & Hardware

IC Resources; United Kingdom; On-site; GBP 85,000 - 140,000; Full time

Job summary

IC Resources is seeking a Senior Software Engineer for an AI engineering organisation to deploy and optimise a leading open-source LLM inference engine across two core programmes: a scaling inference initiative and a high-assurance/regulatory environments initiative.

You will work with Python and low-level languages (C/C++, Rust, CUDA), contribute to upstream OSS, and optimise performance on GPUs/TPUs, with a focus on safety and explainability for regulated settings.

Qualifications

• Senior-level Python and low-level programming (C/C++, Rust, CUDA) experience.
• Contributions to major inference engines or ML infra OSS projects.
• Strong understanding of LLM inference internals (KV caching, batching, quantisation).
• Experience upstreaming to OSS communities.
• Hardware accelerator optimization experience (GPU/TPU).

Responsibilities

• Deploy, instrument, and monitor models served by the inference engine.
• Build engine features for new hardware architectures.
• Extend engine for accuracy, explainability, and safety in regulated contexts.
• Contribute upstream to OSS communities.
• Optimise inference latency, throughput, and hardware utilisation.

Skills

Python

C/C++

Rust

CUDA

OSS contributions

Tools

Git

Linux

CUDA toolkit

Job description

IC Resources is seeking a Senior Software Engineer for an AI engineering organisation to deploy and optimise a leading open-source LLM inference engine across two core programmes: a scaling inference initiative and a high-assurance/regulatory environments initiative.

You will work with Python and low-level languages (C/C++, Rust, CUDA), contribute to upstream OSS, and optimise performance on GPUs/TPUs, with a focus on safety and explainability for regulated settings.