Lead ML Inference Systems Engineer (Remote)

Showcify, Inc.; United States; Remote; USD 150,000 - 220,000; Full time

Benefits offered by this job

Equity/options participation

Medical, dental & vision plans

Flexible PTO

Remote-friendly culture

Home office stipend

Job summary

Runpod is hiring a ML Systems Engineer, Inference to lead the performance end-to-end of LLM serving. You will measure, diagnose, and improve latency and cost across models, hardware generations, and workloads in a remote-first environment.

You will ship fixes that directly affect customer experience, working hands-on on modern inference engines and optimization techniques, with opportunities to influence tooling and deployment strategies across Runpod’s platform.

Qualifications

• 5+ years of professional system engineering experience.
• Deep, hands-on experience with vLLM, SGLang (or a comparable serving engine).
• Strong software engineering skills in Python.
• Solid understanding of what drives LLM inference performance: batching, memory, parallelism, latency-vs-throughput.
• Experience with modern inference optimization techniques such as quantization, speculative decoding, or distributed serving.
• Rigor in benchmarking and performance analysis, plus comfort with GPU profiling tools.
• Ability to explain results clearly in writing and turn them into decisions.

Responsibilities

• Define how we measure inference performance (throughput, time to first token, inter-token latency, cost per token) and build tooling for rigorous, repeatable measurements.
• Profile and diagnose performance problems across the serving stack from scheduling to kernels.
• Improve serving efficiency for large models on single-node and multi-node GPU deployments.
• Turn learnings into production-ready runtimes, configurations, and defaults for customers.
• Work with product and infrastructure teams to shape inference offerings on Runpod.
• Keep up with the inference ecosystem and decide what to adopt, build, or contribute back.
• Trace bottlenecks in the serving engine/runtime and implement fixes when simple tuning isn’t enough.

Skills

Python

Benchmarking

Tools

vLLM

SGLang

CUDA

Triton

Job description

Runpod is hiring a ML Systems Engineer, Inference to lead the performance end-to-end of LLM serving. You will measure, diagnose, and improve latency and cost across models, hardware generations, and workloads in a remote-first environment.

You will ship fixes that directly affect customer experience, working hands-on on modern inference engines and optimization techniques, with opportunities to influence tooling and deployment strategies across Runpod’s platform.