LLM Inference Engineer — High-Performance Rust Systems

Socket.dev; Palo Alto (CA); On-site; USD 230,000 - 350,000; Full time

Benefits offered by this job

Equity opportunity (0.5%)

Professional growth

High-impact work

Job summary

Premier Global Links LLC in Palo Alto, CA is seeking a Member of Technical Staff, Inference Systems, to build and optimize a high-performance AI inference platform from the ground up.

The role focuses on LLM inference, model serving, distributed systems, and inference runtime performance. Strong systems engineering skills and hands-on production experience are required; Rust experience is highly valued.

Qualifications

• 2–10 years of experience in backend, distributed systems, or systems engineering.
• Hands-on experience building, operating, or optimizing LLM inference or serving systems.
• Deep understanding of transformer inference internals, including attention, KV cache, batching, and scheduling.
• Experience with production inference engines such as vLLM, SGLang, or TensorRT-LLM.

Responsibilities

• Build and optimize production LLM inference and model-serving systems.
• Develop inference runtime components using Rust and other systems-level technologies.
• Design and implement batching, scheduling, request routing, and serving infrastructure.
• Build and optimize KV cache and prefix caching systems.
• Scale inference workloads across multi-GPU and multi-node environments.
• Profile, benchmark, and optimize latency, throughput, reliability, and cost.
• Work with inference engines such as vLLM, SGLang, or TensorRT-LLM.
• Investigate performance bottlenecks across the inference stack.
• Contribute to core architecture and technical decisions for the platform.
• Collaborate with a small, hands-on engineering team in a fast-paced environment.

Skills

Rust

C++

Go

Python

Education

Bachelor's or Master's in CS/CE

Tools

vLLM

SGLang

TensorRT-LLM

CUDA

Triton

NCCL

Job description

Premier Global Links LLC in Palo Alto, CA is seeking a Member of Technical Staff, Inference Systems, to build and optimize a high-performance AI inference platform from the ground up.

The role focuses on LLM inference, model serving, distributed systems, and inference runtime performance. Strong systems engineering skills and hands-on production experience are required; Rust experience is highly valued.