Member of Technical Staff, Infrastructure Technical Staff · San Francisco, CA · Full time · On-site

Belvedir; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Belvedir is seeking a Member of Technical Staff, Infrastructure in San Francisco to build and operate the training and serving stack for custom models on Kubernetes, aiming for scalable distributed training, efficient batching, caching, routing, and autoscaling for latency-sensitive deployments.

You will drive down costs per token with quantization, speculative decoding, kernel work, and smarter scheduling while profiling end-to-end bottlenecks across GPU, CPU, network, or model components to

Qualifications

• Experience training or serving LLMs at scale with PyTorch, vLLM, SGLang, or a similar system.
• Strong systems fundamentals: concurrency, memory, networking, and performance measurement.
• Working knowledge of GPU programming (CUDA or Triton) and transformer training/inference internals.
• Fluency in Python plus a systems language (C++, Rust, or Go) and hands-on GPU workloads on Kubernetes.
• Comfort owning production systems and being on the hook when they break.
• Contributions to an open-source inference engine or kernel library.
• Experience with multi-tenant serving, LoRA serving, or model multiplexing.
• Experience at an early-stage startup or as a founder.

Responsibilities

• Build and operate the training and serving stack for custom models on Kubernetes: distributed training, batching, caching, routing, and autoscaling.
• Drive down cost per token with quantization, speculative decoding, kernel work, and smarter scheduling.
• Profile end to end and fix the bottleneck, wherever it is: GPU, CPU, network, or the model itself.
• Set and hit latency and reliability targets for customer deployments.
• Partner with training on model architectures and formats that are cheap to serve.

Skills

LLM infra training/serving

Systems fundamentals

GPU programming (CUDA/Triton)

Python & systems languages

GPU workloads on Kubernetes

Production systems ownership

Open-source contributions

Multi-tenant/LoRA serving

Startup experience

Job description

Member of Technical Staff, Infrastructure

Location San Francisco, CA

Employment Type Full time

Location Type On-site

Department Technical Staff

About Belvedir

Belvedir's mission is to make the creation and deployment of AI models accessible to everyone. To achieve this we build autonomous systems for cleaning raw data, creating benchmarks, asynchronous RL, test-time training, inference optimization, and much more.

We work with startups, enterprises, and some of the world's leading applied AI companies. We have raised a seed round from an amazing group of notable VCs and angels.

About the role

• Build and operate the training and serving stack for custom models on Kubernetes: distributed training, batching, caching, routing, and autoscaling.
• Drive down cost per token with quantization, speculative decoding, kernel work, and smarter scheduling.
• Profile end to end and fix the bottleneck, wherever it is: GPU, CPU, network, or the model itself.
• Set and hit latency and reliability targets for customer deployments.
• Partner with training on model architectures and formats that are cheap to serve.

What we are looking for

• Experience training or serving LLMs at scale with PyTorch, vLLM, SGLang, or a system you built.
• Strong systems fundamentals: concurrency, memory, networking, and performance measurement.
• Working knowledge of GPU programming (CUDA or Triton) and transformer training and inference internals.
• Fluency in Python plus a systems language such as C++, Rust, or Go, and hands-on experience running GPU workloads on Kubernetes.
• Comfort owning production systems and being on the hook when they break.
• Contributions to an open-source inference engine or kernel library.
• Experience with multi-tenant serving, LoRA serving, or model multiplexing.
• Experience at an early-stage startup or as a founder.