Senior Inference Systems Engineer - Rust & GPUs

Wintermeyer Ventures

Palo Alto (CA)

On-site

USD 230,000 - 350,000

Full time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Wintermeyer Ventures is hiring for an experienced backend/distributed systems engineer to help build a high-performance inference platform for large language models. You will shape runtime architecture, optimize KV caching, and scale serving across GPUs and nodes.

The role requires hands-on experience with LLM inference systems, transformer internals, and production engines like vLLM, SGLang, or TensorRT-LLM. Strong systems programming skills in Rust, C++, Go, or Python are essential.

Qualifications

  • Two to ten years of backend or distributed systems engineering experience.
  • Hands-on experience building, operating, or optimizing LLM inference or serving systems, beyond using hosted APIs.
  • Strong understanding of transformer inference, including attention, KV cache, batching, scheduling, and performance bottlenecks.
  • Experience with production inference engines such as vLLM, SGLang, or TensorRT-LLM, and performance-critical systems where latency, throughput, and cost matter.
  • Strong systems programming skills in Rust, C++, Go, or systems-level Python and PyTorch, with the ability to become productive in Rust quickly.

Responsibilities

  • Build a Rust inference runtime, owning batching, scheduling, request routing, and the serving stack.
  • Design KV cache management, prefix caching, and other optimizations to reduce latency and cost per token.
  • Scale inference across multiple GPUs and nodes.
  • Profile and benchmark the inference pipeline, then deliver performance improvements.
  • Collaborate with the team on core architectural decisions.

Skills

Backend engineering
Distributed systems
Rust
C++
Go
Python
PyTorch
Performance tuning
LLM inference
System design

Tools

vLLM
SGLang
TensorRT-LLM
Triton
CUDA

Job description

Wintermeyer Ventures is hiring for an experienced backend/distributed systems engineer to help build a high-performance inference platform for large language models. You will shape runtime architecture, optimize KV caching, and scale serving across GPUs and nodes.

The role requires hands-on experience with LLM inference systems, transformer internals, and production engines like vLLM, SGLang, or TensorRT-LLM. Strong systems programming skills in Rust, C++, Go, or Python are essential.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Member of Technical Staff, Inference Systems
Member of Technical Staff, Inference Systems

Wintermeyer Ventures • Palo Alto (CA)

On-site
USD 230,000 - 350,000
Rust-powered ML Systems Engineer - GPU Inference
Rust-powered ML Systems Engineer - GPU Inference

Pivots Hiring • San Mateo (CA)

On-site
USD 190,000 - 260,000
Senior ML Systems Engineer - Real-Time Inference & Scale
Senior ML Systems Engineer - Real-Time Inference & Scale

BITKRAFT Ventures • Mountain View (CA)

On-site
USD 270,000 - 430,000
Relocation assistance
Equity
Benefits
Senior System Engineer - Agentic AI Inference Equity
Senior System Engineer - Agentic AI Inference Equity

NVIDIA • United States

Hybrid
USD 272,000 - 431,000
Equity
Benefits
Senior Systems Engineer, Inference Runtime
Senior Systems Engineer, Inference Runtime

FDE Times • San Francisco (CA)

On-site
USD 180,000 - 280,000
Senior Inference SW Engineer
Senior Inference SW Engineer

Hewlett Packard Enterprise • Fort Collins (CO)

On-site
USD 180,000 - 240,000
Health And Wellbeing Benefits
Personal And Professional Development
Senior Inference SW Engineer
Senior Inference SW Engineer

Hewlett Packard Enterprise • Durham (NC)

On-site
USD 140,000 - 180,000
Health And Wellbeing Benefits
Personal And Professional Development
AI Inference Engineer (GPU/Rust/CUDA)
AI Inference Engineer (GPU/Rust/CUDA)

Perplexity • New York (NY)

On-site
USD 220,000 - 485,000
Senior Inference Systems Engineer
Senior Inference Systems Engineer

SpaceX Super Intelligence • Palo Alto (CA)

On-site
USD 180,000 - 440,000
Equity
Medical, Vision, Dental
401(k)
+3
Tech Lead: LLM Inference & High-Performance Systems
Tech Lead: LLM Inference & High-Performance Systems

wekatest • United States

Remote
USD 180,000 - 240,000
Medical insurance
Dental insurance
Vision insurance
+5