Senior ML Runtime Engineer for Scalable Inference

Fractile; Bristol; Hybrid; GBP 70,000 - 90,000; Full time

Benefits offered by this job

Competitive salary and equity

Hybrid working

Visible and valued contributions

Job summary

Fractile is looking for a Senior ML Runtime Engineer in the UK to integrate AI acceleration hardware with inference frameworks. Join a small expert team to tackle complex problems like KV cache management and scalable multi-user inference.

This hybrid role offers competitive salary and equity, along with a culture that values learning and collaboration. You'll have the opportunity to shape the runtime stack of cutting-edge AI technologies while working in offices located in London and Bristol.

Qualifications

• Solid experience with ML inference at scale, including multi-user serving.
• A deep understanding of paged attention and inference engines such as vLLM.
• Strong software engineering skills and an instinct for clean, maintainable systems.

Responsibilities

• Integrate Fractile's AI acceleration hardware with leading inference engines.
• Research KV cache management technologies and build proof-of-concept implementations.
• Work closely with the runtime team to design and build a scalable inference engine.
• Focus primarily on the transformer ML architecture.

Skills

ML inference at scale

Paged attention

Inference engines like vLLM

Software engineering

Education

Degree in Computer Science or related field

Tools

Rust

Job description

Fractile is looking for a Senior ML Runtime Engineer in the UK to integrate AI acceleration hardware with inference frameworks. Join a small expert team to tackle complex problems like KV cache management and scalable multi-user inference.

This hybrid role offers competitive salary and equity, along with a culture that values learning and collaboration. You'll have the opportunity to shape the runtime stack of cutting-edge AI technologies while working in offices located in London and Bristol.