Senior ML Inference Engineer - Remote, Equity

Cast AI; Italy; Remote; EUR 160,000 - 231,000; Full time

Benefits offered by this job

Equity options

Remote-first global environment

Learning budget for professional and个人

Annual hackathon and team events

Job summary

Cast AI, a Miami-headquartered, globally distributed team, seeks an experienced ML systems engineer to own inference stack optimization. You will improve throughput, latency, and cost per token across multi-GPU clusters, shaping the technical direction with data-driven experiments.

Your work spans production-grade Python services, tuning SGLang, vLLM, TensorRT-LLM, and CUDA tooling, with a focus on scalable, cost-efficient inference on customer workloads.

Qualifications

• 5+ years building real ML systems for inference or training infrastructure.
• Production-grade Python services, not scripts.
• Hands-on with vLLM, SGLang, or TensorRT-LLM.
• Experience with quantization and KV caches.
• Multi-GPU and multi-node setups with tensor/expert parallelism.
• Know where data resides (page caches, O_DIRECT, network FS).
• Ability to instrument and optimize with measurable gains.

Responsibilities

• Push throughput with continuous batching, prefill, and scheduler tuning across the engine fleet.
• Cut latency by profiling bottlenecks and optimizing compute, memory, storage, and networking.

Skills

Python production-grade

ML inference architecture

Performance benchmarking

GPU/cluster optimization

Tools

SGLang

vLLM

TensorRT-LLM

PyTorch

CUDA

Kubernetes

Job description

Cast AI, a Miami-headquartered, globally distributed team, seeks an experienced ML systems engineer to own inference stack optimization. You will improve throughput, latency, and cost per token across multi-GPU clusters, shaping the technical direction with data-driven experiments.

Your work spans production-grade Python services, tuning SGLang, vLLM, TensorRT-LLM, and CUDA tooling, with a focus on scalable, cost-efficient inference on customer workloads.