Senior LLM Inference & Serving Engineer

Premier Global Links LLC; Palo Alto, Northern (CA, KY); Hybrid; USD 230,000 - 350,000; Full time

Benefits offered by this job

Equity 0.5%

Job summary

Premier Global Links LLC is seeking an experienced Member of Technical Staff, Inference Systems, to build and optimize a high-performance AI inference platform from the ground up. This role focuses on LLM inference, model serving, distributed systems, and inference runtime performance in Palo Alto.

You will work with Rust and other systems-level technologies to design batching, routing, and caching, and to scale workloads across multi-GPU and multi-node environments.

Qualifications

• 2–10 years of backend or systems engineering experience.
• Hands-on experience with LLM inference or serving systems.
• Deep understanding of transformer inference internals (attention, KV cache, batching).
• Experience with production inference engines (vLLM, SGLang, TensorRT-LLM).
• Strong Rust, C++, Go, or systems-level Python/PyTorch for systems work.

Responsibilities

• Build and optimize production LLM inference and model-serving systems.
• Develop inference runtime components using Rust and other systems-level technologies.
• Design and implement batching, scheduling, request routing, and serving infrastructure.
• Build and optimize KV cache and prefix caching systems.
• Scale inference workloads across multi-GPU and multi-node environments.
• Profile, benchmark, and optimize latency, throughput, reliability, and cost.
• Work with inference engines such as vLLM, SGLang, or TensorRT-LLM.
• Investigate performance bottlenecks across the inference stack.
• Contribute to core architecture and technical decisions for the platform.
• Collaborate with a small, hands-on engineering team in a fast-paced environment.

Skills

LLM inference

Distributed systems

Rust

Model serving

Performance optimization

Python/PyTorch (systems-level)

Education

Bachelor's or Master's in CS/CE

Relevant graduate degree preferred

Tools

vLLM

SGLang

TensorRT-LLM

CUDA

Triton

Job description

Premier Global Links LLC is seeking an experienced Member of Technical Staff, Inference Systems, to build and optimize a high-performance AI inference platform from the ground up. This role focuses on LLM inference, model serving, distributed systems, and inference runtime performance in Palo Alto.

You will work with Rust and other systems-level technologies to design batching, routing, and caching, and to scale workloads across multi-GPU and multi-node environments.