Senior AI Inference Systems Engineer

StackYak

San Francisco (CA)

Remote

USD 180,000 - 280,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

StackYak is building the infrastructure layer for AI in San Francisco. This is an early-stage founding role where you will shape the systems layer beneath the product, with broad autonomy and rapid execution.

You will own how models run, choosing precision, quantization, and memory strategies, and selecting serving runtimes. You will benchmark, deploy, and operate real inference systems at scale, influencing architecture and automation across multiple GPUs and nodes.

Qualifications

  • Experience running large-scale inference in production.
  • Ability to deploy models across multiple GPUs and nodes.
  • Proven track record in latency, throughput, and cost trade-offs.

Responsibilities

  • Run production inference systems with real customer workloads.
  • Benchmark, deploy, debug, and operate inference infrastructure at scale.

Skills

Inference infra
Python in prod
GPU deployment
NVIDIA/AMD inference

Tools

vLLM
TensorRT-LLM
SGLang

Job description

StackYak is building the infrastructure layer for AI in San Francisco. This is an early-stage founding role where you will shape the systems layer beneath the product, with broad autonomy and rapid execution.

You will own how models run, choosing precision, quantization, and memory strategies, and selecting serving runtimes. You will benchmark, deploy, and operate real inference systems at scale, influencing architecture and automation across multiple GPUs and nodes.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Systems Engineer, AI Inference Platform
Senior Systems Engineer, AI Inference Platform

OpenAI, Inc. • San Francisco (CA)

On-site
USD 180,000 - 260,000
Staff AI Inference Systems Engineer
Staff AI Inference Systems Engineer

Anthropic Limited • San Francisco (CA)

On-site
USD 320,000 - 485,000
Equity donation matching
Generous vacation
Parental leave
+2
Senior AI Inference Engineer — Scale & Production
Senior AI Inference Engineer — Scale & Production

Spectro Cloud • San Jose (CA)

On-site
USD 140,000 - 210,000
Senior Backend Engineer, Inference Platform (Equity)
Senior Backend Engineer, Inference Platform (Equity)

Together Computer Inc • San Francisco (CA)

On-site
USD 200,000 - 290,000
Equity
Health insurance
Competitive base salary
Inference Systems Engineer — High-Performance AI Serving
Inference Systems Engineer — High-Performance AI Serving

River AI • Palo Alto (CA)

On-site
USD 200,000 - 420,000
Health insurance
Dental insurance
Vision insurance
+3
Head of AI Inference & Systems Platform
Head of AI Inference & Systems Platform

Blackhornvc • San Francisco (CA)

On-site
USD 220,000 - 420,000
Infrastructure Inference Engineer for Scalable AI
Infrastructure Inference Engineer for Scalable AI

Bison • San Francisco (CA)

On-site
USD 400,000 - 500,000
Health, dental, and vision benefits
Unlimited PTO
Paid parental leave
+1
Senior Distributed Systems Engineer — AI Inference Platform
Senior Distributed Systems Engineer — AI Inference Platform

Baseten • San Francisco (CA)

On-site
USD 180,000 - 240,000
Equity
Comprehensive medical for US
Flexible PTO
+4
Senior AI Inference Systems Engineer
Senior AI Inference Systems Engineer

Cloudflare, Inc. • New York (NY)

Hybrid
USD 185,000 - 254,000
Equity plan
Health insurance
401(k) Retirement Savings Plan
+1
Senior AI Platform Engineer (Hybrid)
Senior AI Platform Engineer (Hybrid)

People in AI • San Francisco (CA)

Hybrid
USD 180,000 - 240,000