Senior Inference Engineer — GPU-Disaggregated Serving

TensorX; Dublin; Hybrid; EUR 120,000 - 180,000; Full time

Benefits offered by this job

Hybrid working

25 days paid annual leave

Free inference tokens!

Job summary

TensorX, a Dublin-based sovereign AI infrastructure platform, seeks a Senior Inference Engineer to own the disaggregated serving layer across multiple sites. You will manage admission, routing, and autoscaling for high-concurrency workloads, ensuring low latency and zero data retention.

You will collaborate with GPU Performance, Platform, and Backend teams to optimize CUDA workloads, pipelines, and Kubernetes clusters, while delivering robust, scalable serving.

Qualifications

• 5+ years in distributed systems, ML infra, or production serving; GPU workloads a plus.
• Hands-on experience running LLMs in production with vLLM, SGLang or TensorRT-LLM.
• Experience with NVIDIA Dynamo or comparable disaggregated serving (llm-d).
• Kubernetes in production: deployments, rollouts, operators, or GPU scheduling.
• Strong fundamentals in queues, backpressure, routing, and burst-load retries.
• Proficiency in Python and reading systems code; comfortable with AI-assisted tools.

Responsibilities

• Disaggregated serving: extend Dynamo deployment; manage prefill/decode split and cross-node capacity.
• Admission & routing: own router and KV-aware routing; implement burst queues and cache-tuning collaboration.
• Autoscaling: scale prefill/decode to meet latency targets; decide GPU allocation per model pool.

Skills

Distributed systems

GPU workloads

Python

Performance tuning

Cloud/On-prem infra

Education

BSc/MSc in Computer Science, Software Engineering, Electrical Engineering or related field

Tools

Kubernetes

Prometheus

Grafana

Loki

Job description

TensorX, a Dublin-based sovereign AI infrastructure platform, seeks a Senior Inference Engineer to own the disaggregated serving layer across multiple sites. You will manage admission, routing, and autoscaling for high-concurrency workloads, ensuring low latency and zero data retention.

You will collaborate with GPU Performance, Platform, and Backend teams to optimize CUDA workloads, pipelines, and Kubernetes clusters, while delivering robust, scalable serving.