Hardware-Aware Inference Engine Engineer

Uncover; Greater London; Hybrid; GBP 110,000 - 170,000; Full time

Benefits offered by this job

Competitive salary

Equity & Ownership

Private healthcare

Visa sponsorship and relocation

Job summary

Callosum is seeking a seasoned ML inference engineer to push heterogeneous hardware orchestration beyond GPUs. You will work on SGLang and vLLM, extending them to run efficiently across diverse accelerators, focusing on scheduling, memory and execution.

Based in London, you will collaborate with an Accelerator Systems Software engineer, contribute upstream, and maintain internal forks while scaling production inference on mixed hardware.

Qualifications

• Deep familiarity with SGLang and vLLM, or comparable inference serving frameworks, including scheduler design, memory management and execution pipelines.
• Strong background in high-performance Python and C++/CUDA systems, particularly in ML inference.
• Experience designing or implementing parallelism strategies for large model serving.
• Understanding of disaggregated serving architectures and tradeoffs in separating modules of a workflow.
• Proven ability to contribute to fast-moving open source codebases with evolving APIs and design conventions.

Responsibilities

• Contribute upstream to SGLang and vLLM, and maintain internal forks where our requirements diverge.
• Improve hardware-awareness within inference engines so that scheduling, memory management, and execution adapt to the capabilities of the underlying accelerator.
• Design and implement bespoke parallelism and disaggregation strategies that go beyond default configurations to better exploit heterogeneous hardware.
• Work closely with an Accelerator Systems Software engineer to ensure engine-level abstractions map cleanly onto diverse hardware capabilities.

Skills

SGLang familiarity / vLLM

Scheduler design

Memory management

C++ / CUDA

Large model serving

Tools

CUDA

Linux

Job description

Callosum is seeking a seasoned ML inference engineer to push heterogeneous hardware orchestration beyond GPUs. You will work on SGLang and vLLM, extending them to run efficiently across diverse accelerators, focusing on scheduling, memory and execution.

Based in London, you will collaborate with an Accelerator Systems Software engineer, contribute upstream, and maintain internal forks while scaling production inference on mixed hardware.