Open-Weight Inference Systems Architect

Gradiant; Greater London; Hybrid; GBP 110,000 - 140,000; Full time

Job summary

Gradiant is seeking an engineer to identify bottlenecks across the inference stack for open-weight models. You will measure targets on NVIDIA and AMD hardware, design kernels and memory layouts, and push changes to runtimes and serving paths.

The role emphasizes turning research into production services and sharing findings publicly. You should be proficient in Python and a systems language (C++ or Rust), with experience in GPU programming or low-level hardware considerations and a strong grasp

Qualifications

• Experience building ML systems, compilers, runtimes, or kernels.
• Ability to explain decisions and results in detail.
• Proficiency in Python and a systems language (C++ or Rust).
• CUDA or HIP kernel writing or strong low-level GPU experience.

Responsibilities

• Measure models and workloads on NVIDIA and AMD hardware.
• Design and implement kernels, formats, memory layouts, and cache policies.
• Modify runtimes, schedulers, and serving paths to improve performance.
• Test latency, throughput, energy use, and model correctness on real workloads.
• Turn research into production services.
• Share research publicly to educate others.
• Collaborate with silicon team to identify bottlenecks and opportunities.

Skills

ML systems

Python

C++ or Rust

CUDA/HIP kernels

Profiling

Transformer execution

Production code

Tools

CUDA

GPU kernels

Job description

Gradiant is seeking an engineer to identify bottlenecks across the inference stack for open-weight models. You will measure targets on NVIDIA and AMD hardware, design kernels and memory layouts, and push changes to runtimes and serving paths.

The role emphasizes turning research into production services and sharing findings publicly. You should be proficient in Python and a systems language (C++ or Rust), with experience in GPU programming or low-level hardware considerations and a strong grasp