ML Runtime Engineer: Scale Inference Engines

Fractile; West of England; Hybrid; GBP 90,000 - 150,000; Full time

Benefits offered by this job

Equity

Private medical

Dental & Vision

Contributory Pension

25 days holiday + bank holidays

Life/Critical Illness Insurance

Job summary

Fractile is hiring a ML Runtime Engineer to integrate our AI accelerators with the latest inference frameworks and to build a scalable runtime stack. You will tackle challenges such as KV cache management and multi-user inference, focusing on transformer architectures in a collaborative environment.

The role requires deep experience in ML inference at scale, knowledge of paged attention and vLLM, and strong software engineering skills to maintain robust, high-performance systems.

Qualifications

• Experience with ML inference at scale.
• Deep understanding of paged attention and inference engines such as vLLM.
• Familiarity with key components of the ML software ecosystem.
• Strong software engineering skills and clean, maintainable systems.

Responsibilities

• Integrate Fractile's AI accelerators with leading inference engines.
• Research and implement KV cache management tailored to hardware.
• Design and build a scalable reference inference engine.
• Collaborate with the runtime team on transformer ML architectures.

Skills

ML inference at scale

Paged attention

vLLM familiarity

Software engineering

Education

Degree in Computer Science

Tools

Rust

Job description

Fractile is hiring a ML Runtime Engineer to integrate our AI accelerators with the latest inference frameworks and to build a scalable runtime stack. You will tackle challenges such as KV cache management and multi-user inference, focusing on transformer architectures in a collaborative environment.

The role requires deep experience in ML inference at scale, knowledge of paged attention and vLLM, and strong software engineering skills to maintain robust, high-performance systems.