Senior ML Inference Engineer — Remote-First, Throughput Mastery

Cast AI

Polska

Remote

PLN 466,000 - 700,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Equity options
Remote-friendly
Learning budget
Annual hackathon
Equipment budget
Extra days off

Job summary

Cast AI, a Miami-headquartered, global team, seeks an experienced ML infrastructure engineer to optimize inference workloads. You will tune throughput, reduce latency, and advance KV cache strategies while managing multi-GPU, multi-node deployments.

You’ll work with vLLM, SGLang, and TensorRT-LLM in a remote-first environment, contributing to performance experiments and scalable systems used across customers.

Qualifications

  • 5+ years building ML systems for inference or training infrastructure.
  • Production-grade Python services, not scripts.
  • Hands-on with vLLM, SGLang, or TensorRT-LLM on real workloads.
  • Experience with quantization and performance measurement.
  • Multi-GPU and multi-node setups with parallelism and communication.

Responsibilities

  • Push throughput with continuous batching and scheduler tuning.
  • Cut latency by profiling bottlenecks across compute, memory, and I/O.
  • Improve KV cache performance with caching and quantization strategies.
  • Reduce cold starts via optimized loaders and JIT caching.
  • Quantize without sacrificing model quality across weights and activations.
  • Scale inference across nodes with distributed topologies.
  • Set and communicate the technical direction with reproducible experiments.

Skills

Inference infra
Python production
vLLM
SGLang
TensorRT-LLM
Multi-GPU
Quantization
Kubernetes
Profiling
Distributed systems

Tools

PyTorch
CUDA
Kubernetes
ClickHouse
Prometheus
Grafana
GitLab CI
OpenTelemetry
fastsafetensors
TensorRT-LLM
vLLM
SGLang

Job description

Cast AI, a Miami-headquartered, global team, seeks an experienced ML infrastructure engineer to optimize inference workloads. You will tune throughput, reduce latency, and advance KV cache strategies while managing multi-GPU, multi-node deployments.

You’ll work with vLLM, SGLang, and TensorRT-LLM in a remote-first environment, contributing to performance experiments and scalable systems used across customers.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior ML Ops Engineer — Remote GPU Inference & Scale
Senior ML Ops Engineer — Remote GPU Inference & Scale

Pragmatike • Poland

Remote
PLN 301,000 - 474,000
Remote AI Infrastructure Engineer - GPU/ML Ops
Remote AI Infrastructure Engineer - GPU/ML Ops

Pragmatike • Poland

Remote
PLN 364,033 - 473,242
Ownership of critical infrastructure
High-growth startup environment
Inclusive recruitment process
Senior ML Engineer – CV/LLM, Remote & Flexible Hours
Senior ML Engineer – CV/LLM, Remote & Flexible Hours

FLYPS • Poland

Remote
PLN 290,000 - 368,000
20% remote setup
Flexible hours
Private medical care
+3
CUDA & AI Engineer — Flexible Hours & Growth
CUDA & AI Engineer — Flexible Hours & Growth

MWDN Ltd. • Województwo mazowieckie

Remote
PLN 120,000 - 180,000
PTO 29 days
Free English classes
HR support
+1
AI Infrastructure Engineer (GPU) - Remote EMEA
AI Infrastructure Engineer (GPU) - Remote EMEA

Pragmatike • Poland

Remote
PLN 364,033 - 473,242
Ownership of critical infrastructure
High-growth startup environment
Inclusive recruitment process
Senior AIOps Engineer — ML Platform & GPU Infra
Senior AIOps Engineer — ML Platform & GPU Infra

CSHARK • Wrocław

Hybrid
PLN 120,000 - 180,000
Remote-first work environment
Flexible hours
International projects
+1
Inference Engine Engineer - Scalable AI Serving
Inference Engine Engineer - Scalable AI Serving

Mistral • Warszawa

Hybrid
PLN 180,000 - 300,000
Senior ML & AI Engineer — GenAI & LLMs, Remote
Senior ML & AI Engineer — GenAI & LLMs, Remote

SPD Technology • Poland

Remote
PLN 180,000 - 280,000
Fully remote work
Flexible schedule
Provided hardware and software
Backend AI Infra Engineer — Remote, Equity, High Impact
Backend AI Infra Engineer — Remote, Equity, High Impact

Sowelo Consulting • Poland

Remote
PLN 180,000 - 260,000
Fully remote
Equity
Production ML Engineer: Real-Time Inference & MLOps
Production ML Engineer: Real-Time Inference & MLOps

Empowers Staffing Inc. • Polska

Hybrid
PLN 210,000 - 310,000
Top locations