Lead ML Inference Systems Engineer (Remote)

Runpod; España; Remote; EUR 134,000 - 196,000; Full time

Benefits offered by this job

Equity

Remote work

Stock options

Healthcare plan

Flexible PTO

Job summary

Runpod is hiring a ML Systems Engineer, Inference, to lead end-to-end LLM serving performance and cost optimization. You’ll measure, diagnose, and improve throughput, latency, and efficiency across models and hardware generations in production.

You’ll own tooling, runtimes, and configurations, collaborating with product and infra teams to shape inference offerings while staying current with the open-source ecosystem and benchmarking best practices.

Qualifications

• 5+ years of professional system engineering experience.
• Deep, hands-on experience with vLLM, SGLang (or a comparable serving engine) in production or at serious benchmark scale.
• Strong software engineering skills in Python. You're comfortable working in large, performance-critical codebases.
• A solid understanding of what drives LLM inference performance: batching, memory, parallelism, and the trade-offs between latency and throughput.
• Experience with modern inference optimization techniques such as quantization, speculative decoding, or distributed serving.
• Rigor in benchmarking and performance analysis, plus comfort with GPU profiling tools.
• The ability to explain your results clearly in writing and turn them into decisions.

Responsibilities

• Define how we measure inference performance, including throughput, time to first token, inter-token latency, and cost per token, and build the tooling that makes those measurements rigorous and repeatable.
• Profile and diagnose performance problems across the serving stack, from scheduling and memory management down to kernels and interconnect.
• Improve serving efficiency for large, state-of-the-art models on single-node and multi-node GPU deployments.
• Turn what you learn into production‑ready runtimes, configurations, and defaults that customers benefit from automatically.
• Work closely with product and infrastructure teams to shape how inference is offered on Runpod.
• Keep up with the fast-moving inference ecosystem, including the open‑source community, and decide what's worth adopting, what's worth building, and what's worth contributing back.
• Trace bottlenecks in the serving engine/runtime and implement fixes when configuration tuning is not enough.

Skills

Python

System engineering

Benchmarking

GPU profiling

Performance tuning

Inference optimization

Tools

vLLM

SGLang

Job description

Runpod is hiring a ML Systems Engineer, Inference, to lead end-to-end LLM serving performance and cost optimization. You’ll measure, diagnose, and improve throughput, latency, and efficiency across models and hardware generations in production.

You’ll own tooling, runtimes, and configurations, collaborating with product and infra teams to shape inference offerings while staying current with the open-source ecosystem and benchmarking best practices.