Remote Senior ML Inference Systems Engineer

Runpod; Santa Cruz de Tenerife; Remote; EUR 134,000 - 196,000; Full time

Benefits offered by this job

Equity

Medical/Dental/Vision

Flexible PTO

Home Office stipend

Job summary

Runpod is the AI Developer Cloud, powering over a million developers to run LLM inference with focus on speed and cost efficiency. This hands-on role leads performance end-to-end across models, hardware generations, and workloads, delivering production-ready runtimes and configurations.

You will work with product and infra teams to shape Runpod's inference offering, stay current with open-source trends, and drive improvements that customers experience in latency and cost.

Qualifications

• 5+ years of professional system engineering experience.
• Deep, hands-on experience with vLLM, SGLang (or a comparable serving engine) in production or benchmark scale.
• Strong software engineering skills in Python. Comfortable in large, performance-critical codebases.
• Solid understanding of what drives LLM inference performance: batching, memory, parallelism, latency vs throughput.
• Experience with modern inference optimization techniques such as quantization, speculative decoding, or distributed serving.
• Rigor in benchmarking and performance analysis; comfort with GPU profiling tools.
• Ability to explain results in writing and turn them into decisions.

Responsibilities

• Define how we measure inference performance (throughput, time to first token, inter-token latency, cost per token) and build rigorous tooling.
• Profile and diagnose performance issues across the serving stack (scheduling, memory management, kernels, interconnect).
• Improve serving efficiency for large, state-of-the-art models on single-node and multi-node GPU deployments.
• Turn learnings into production-ready runtimes, configurations, and defaults for customers.
• Collaborate with product and infra teams to shape inference offering on Runpod.
• Keep up with the inference ecosystem and decide what to adopt or build; contribute back when appropriate.
• Trace bottlenecks and implement fixes when tuning is not enough.

Skills

Python

vLLM

SGLang

Performance profiling

Tools

CUDA

Triton

GPU profiling tools

Job description

Runpod is the AI Developer Cloud, powering over a million developers to run LLM inference with focus on speed and cost efficiency. This hands-on role leads performance end-to-end across models, hardware generations, and workloads, delivering production-ready runtimes and configurations.

You will work with product and infra teams to shape Runpod's inference offering, stay current with open-source trends, and drive improvements that customers experience in latency and cost.