Senior Inference Engineer - Production Focus (Remote)

Vcluster; San Francisco (CA); Hybrid; USD 190,000 - 220,000; Full time

Benefits offered by this job

Equity

Health, dental, vision, and life保险

Flexible working schedule

Workplace flexibility

Job summary

vCluster Labs is seeking a Senior Inference Engineer to own the platform's inference layer from the ground up. You will deploy LLMs to production, run serving frameworks, and optimize for scale while building production-grade code in Python or Golang.

You’ll work closely with the CTO and Product to shape the roadmap and ensure robust, scalable inference across multiple GPU machines. The role emphasizes hands-on programming, production readiness, and strong communication to align engineers and

Qualifications

• Production LLMs serving experience.
• Experience with quantization, batching, caching, and routing.
• Strong programming in Python or Golang with production code.
• Familiarity with containerized environments (Docker, Kubernetes).
• Generative AI exposure with PyTorch or Transformers.
• Good understanding of GPU stack (CUDA, NCCL, drivers).
• Knowledge of model architectures and fine-tuning approaches.
• Experience with NVIDIA Dynamo.

Responsibilities

• Deploy models to production across GPU infrastructure, owning the full pipeline from query to response.
• Stand up and operate serving infrastructure using vLLM, SGLang, or TensorRT-LLM.
• Optimize for latency and cost via quantization, batching, caching, and routing.
• Build production-grade infrastructure in Python or Golang.

Skills

LLM Serving

Python

Golang

Communication

Tools

Docker

Kubernetes

CUDA

NCCL

NVIDIA Dynamo

Job description

vCluster Labs is seeking a Senior Inference Engineer to own the platform's inference layer from the ground up. You will deploy LLMs to production, run serving frameworks, and optimize for scale while building production-grade code in Python or Golang.

You’ll work closely with the CTO and Product to shape the roadmap and ensure robust, scalable inference across multiple GPU machines. The role emphasizes hands-on programming, production readiness, and strong communication to align engineers and