Staff Infra Engineer – LLM Inference

Morgan Pinnacle Group; San Francisco (CA); On-site; USD 350,000 - 450,000; Full time

Job summary

Morgan Pinnacle Group is seeking an Infrastructure Engineer to lead production LLM inference infrastructure onsite in San Francisco. The role emphasizes building and operating scalable Kubernetes clusters across AWS, GCP, and GPU cloud providers to support high-throughput inference workloads.

You will own the production inference serving layer, debug GPU driver stacks, and partner with model engineers to ensure performance gains translate to production.

Qualifications

• 5+ years of infrastructure engineering experience.
• Direct, day-to-day experience with production LLM inference infrastructure.
• Extensive experience operating production Kubernetes at scale across multiple cloud providers (AWS, GCP).
• Hands-on expertise debugging NVIDIA GPU and accelerator systems (drivers, CUDA, NCCL, node health, networking).
• Experience with at least one inference serving framework (e.g., vLLM, SGLang, TensorRT-LLM, NVIDIA Dynamo).
• Experience with high-performance networking (InfiniBand, RDMA/EFA, MPI).
• Strong observability and reliability engineering background for latency-sensitive services.
• Ability to diagnose and resolve complex production infrastructure issues.
• Strong communication and collaboration with customers and technical stakeholders.
• Ability to work onsite in San Francisco five days per week.

Responsibilities

• Design, operate, and scale highly available Kubernetes clusters across AWS, GCP, and GPU cloud providers for inference workloads >1,000 tokens/sec.
• Own the production inference serving layer; debug NVIDIA drivers, CUDA, NCCL, node health, and network fabric.
• Collaborate with model optimization engineers to ensure performance translates to production.
• Extend deployment tooling for consistent model deployment across NVIDIA GPUs, AWS Trainium, and Google TPUs.
• Build and maintain observability systems tracking tft, inter-token latency, throughput, and availability.
• Lead reliability engineering with monitoring, alerting, auto-failover, self-healing infra, and traffic routing.
• Troubleshoot production infrastructure issues directly with customers and technical teams.

Skills

5+ years experience

LLM inference infra

Observability & reliability

Communication & problem-solving

Onsite in SF

Tools

Kubernetes

AWS

GCP

NVIDIA GPUs

NCCL

CUDA

vLLM

SGLang

TensorRT-LLM

NVIDIA Dynamo

InfiniBand / RDMA

MPI

Job description

Morgan Pinnacle Group is seeking an Infrastructure Engineer to lead production LLM inference infrastructure onsite in San Francisco. The role emphasizes building and operating scalable Kubernetes clusters across AWS, GCP, and GPU cloud providers to support high-throughput inference workloads.

You will own the production inference serving layer, debug GPU driver stacks, and partner with model engineers to ensure performance gains translate to production.