LLM Inference Systems Engineer

Netpreme; Santa Clara, Boston (CA, MA); On-site; USD 170,000 - 210,000; Full time

Benefits offered by this job

Competitive salary with bonus

Health, Dental, and Vision for you and

401(k) match with immediate vesting

Life, Disability, and AD&D insurance

PTO: 20 days + 15 holidays

Lunch stipend

Office in Santa Clara, CA and Boston,

Relocation assistance

Visa sponsorship

Collaborative learning environment

Job summary

Netpreme is seeking a skilled engineer to own the inference-serving stack for frontier LLM workloads. You will optimize and operate distributed inference pipelines, working with vLLM, SGLang, and TensorRT-LLM, while coordinating with hardware and systems teams.

You’ll build benchmarks, tune performance, and drive production readiness with Kubernetes, NVIDIA Dynamo, Helm, and Ansible, in a fast-moving startup setting.

Qualifications

• BS, MS, or PhD in Computer Science, Computer Engineering, or a related field, or equivalent practical experience.
• 2+ years of engineering experience in LLM inference, ML systems, GPU systems, or related infrastructure.
• Hands-on experience deploying and optimizing at least one LLM inference engine (e.g., vLLM, SGLang, or TensorRT-LLM).
• Solid understanding of LLM inference fundamentals: prefill/decode, batching, KV caching, and latency/throughput tradeoffs.
• Strong Python and Linux skills, plus hands-on experience with Kubernetes and containerized deployments.
• Comfortable benchmarking and troubleshooting systems, and communicating clearly in a small engineering team.

Responsibilities

• Deploy and operate models in production: Run large language and multimodal models using inference engines like vLLM, SGLang, and TensorRT-LLM.
• Own the serving infrastructure: Build and maintain distributed inference systems with NVIDIA Dynamo, Kubernetes, Helm, and Ansible, including upgrades, monitoring, troubleshooting, and reliability.
• Optimize performance: Tune parallelism strategies (TP/EP/DP/PP), batching, KV caching, routing, and scheduling to meet latency, throughput, and SLA targets.
• Benchmark and measure: Build reproducible benchmarks for latency (TTFT, TPOT, E2E), throughput, concurrency scaling, and system efficiency.
• Support diverse workloads: Enable tool calling, agentic and coding workflows, RAG, and long-context applications.
• Bring new tech into production: Evaluate new models, frameworks, and optimizations, and partner with hardware, model, and systems engineers to ship improvements.

Skills

Python

Linux

Education

BS/MS/PhD in CS/CE

Tools

vLLM

SGLang

TensorRT-LLM

Kubernetes

Helm

Ansible

NVIDIA Dynamo

Job description

Netpreme is seeking a skilled engineer to own the inference-serving stack for frontier LLM workloads. You will optimize and operate distributed inference pipelines, working with vLLM, SGLang, and TensorRT-LLM, while coordinating with hardware and systems teams.

You’ll build benchmarks, tune performance, and drive production readiness with Kubernetes, NVIDIA Dynamo, Helm, and Ansible, in a fast-moving startup setting.