Senior ML Inference Systems Engineer | Remote & Equity

Cast AI; France; Remote; EUR 90,000 - 140,000; Full time

Benefits offered by this job

Equity options

Remote-first

Learning budget

Annual hackathon

Team events

Equipment budget

Extra days off

Job summary

Cast AI is seeking a senior ML inference infrastructure engineer to optimize throughput, latency, and cost across our GPU fleet. You will tune loaders, schedulers, and KV caches, ensuring scalable, multi-node inference while maintaining model quality.

You will own the technical direction, work with vLLM/SGLang/TensorRT-LLM stacks, and collaborate across a global, remote-first team. A strong production background is essential.

Qualifications

• You've built real ML systems for 5+ years, with depth in inference or training infrastructure.
• Your Python is production-grade: services, not scripts.
• You've run vLLM, SGLang, or TensorRT-LLM for real and understand performance.
• You've measured quantization regressions, not just compression ratios.
• Multi-GPU and multi-node setups are familiar: tensor/expert parallelism and communication.
• You instrument before you optimize and can identify real wins from benchmarks.

Responsibilities

• Push throughput: continuous batching, prefill, drafting models, CUDA-graph capture, scheduler tuning across the engine fleet.
• Cut latency: profile to find bottlenecks and fix compute, memory, scheduling, or storage issues.
• Get more out of the KV cache: hierarchical/paged attention, prefix caching, eviction policies, quantized KV.
• Kill cold starts: weight loaders, JIT and compile caches, shared-filesystem paths.
• Quantize without regressing quality: INT8/INT4, FP8/FP4 across weights/activations/KV.
• Scale across nodes: distributed inference, network-aware placement, checkpointing strategies.

Skills

Python production-grade

ML inference systems

vLLM/SGLang/TensorRT-LLM experience

Quantization awareness

Multi-GPU/cluster knowledge

Performance profiling

System design & optimization

Tools

SGLang

vLLM

TensorRT-LLM

PyTorch

Kubernetes

CUDA tooling

ClickHouse

OpenTelemetry

Prometheus

Grafana

GitLab CI

Job description

Cast AI is seeking a senior ML inference infrastructure engineer to optimize throughput, latency, and cost across our GPU fleet. You will tune loaders, schedulers, and KV caches, ensuring scalable, multi-node inference while maintaining model quality.

You will own the technical direction, work with vLLM/SGLang/TensorRT-LLM stacks, and collaborate across a global, remote-first team. A strong production background is essential.