Senior Inference Engineer - Hybrid, Free Inference Tokens

Tensorix Ltd.; Dublin; Hybrid; EUR 120,000 - 160,000; Full time

Benefits offered by this job

Hybrid working

25 days paid annual leave

Free inference tokens!

Job summary

TensorX in Dublin is seeking a Senior Inference Engineer to own the serving layer between the API gateway and inference engines across multiple sites. You will manage disaggregated serving, routing, autoscaling, and KV cache movement while ensuring data privacy and low latency.

You will collaborate with the GPU Performance Team, platform, and backend teams to optimize throughput and reliability for long-context, high-concurrency workloads.

Qualifications

• 5+ years of professional experience in distributed systems, ML infrastructure or production serving.
• Hands-on experience running large language models in production with vLLM, SGLang or TensorRT-LLM.
• Experience with NVIDIA Dynamo or a comparable disaggregated serving system (e.g. llm-d).
• Kubernetes in production, including deployments, rollouts, operators or controllers and GPU scheduling
• Strong distributed systems fundamentals: queueing, backpressure, admission control, routing and retries under burst load
• Experience instrumenting production systems (Prometheus, Grafana, Loki) and using that data to guide tuning decisions
• Familiarity with benchmarking methodology: one variable at a time, a pass or fail bar set before the run and results someone else can reproduce
• Proficiency in Python and comfort reading systems code in other languages
• Comfortable using AI-assisted development tools (e.g. Claude Code, Codex) as part of your daily workflow
• A clear and concise communicator who thrives in ambiguity and can articulate technical decisions to both technical and non-technical audiences

Responsibilities

• Disaggregated serving - Run and extend our NVIDIA Dynamo deployment and manage prefill/decode split.
• Admission & routing - Own router and KV-aware routing; build burst-safe admission control.
• Autoscaling & capacity - Scale prefill and decode to meet latency targets across multi-site workloads.
• KV cache transfer - Own cross-node cache movement and test across nodes with telemetry.
• Multi-site serving - Run several sites behind one front door with local serving planes.
• Observability - Instrument serving with per-model metrics and integrate with Prometheus, Grafana, Loki.
• Rollouts & pre-production - Deploy changes with zero downtime and gate through GPU team.

Skills

Distributed systems

GPU workloads

LLM in prod

Kubernetes

Python

TensorRT-LLM

llm-d

Telemetry

Observability

Communication

Education

BSc/MSc in CS/EE or related

Tools

NVIDIA Dynamo

llm-d

Kubernetes

Prometheus

Grafana

Loki

Job description

TensorX in Dublin is seeking a Senior Inference Engineer to own the serving layer between the API gateway and inference engines across multiple sites. You will manage disaggregated serving, routing, autoscaling, and KV cache movement while ensuring data privacy and low latency.

You will collaborate with the GPU Performance Team, platform, and backend teams to optimize throughput and reliability for long-context, high-concurrency workloads.