Product Manager, AI Inference - Scale & Performance

AItoolnavio

San Francisco (CA)

On-site

USD 150,000 - 190,000

Full time

5 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Equity & competitive compensation
Health insurance (US)
Flexible PTO (Winter Break)
Parental leave
401(k)
Learning opportunities

Job summary

Baseten is hiring a deeply technical Product Manager to accelerate AI inference performance at scale. You will work alongside inference engineers and researchers to turn model-serving advances into tangible customer features that improve latency, throughput, and cost in real production workloads.

You will bridge cutting-edge systems engineering with real-world workloads, helping customers choose optimal configurations and ensuring smooth release cycles.

Qualifications

  • Experience shipping AI/ML products or infrastructure with focus on performance, scalability, or production behavior.
  • Technical background in software engineering, ML engineering, research, or equivalent hands-on experience.
  • Strong understanding of model serving architectures, GPU economics, latency, throughput, and cost tradeoffs.
  • Experience working with researchers or infrastructure engineers to bring complex capabilities into production.
  • Rigorous benchmarking, experimentation, and performance measurement approach.
  • Strong product judgment translating deep technical work into customer value.

Responsibilities

  • Collaborate with inference engineering and research to bring new performance capabilities to customers.
  • Translate advances in batching, caching, quantization, speculative decoding, and serving architectures into product features.
  • Define benchmarks and evaluations that demonstrate meaningful improvements in latency, throughput, quality, and cost.
  • Understand customer workloads and help prioritize optimizations with the greatest impact.
  • Make performance features easier to configure, evaluate, and adopt in production.
  • Help customers understand tradeoffs across hardware, model configurations, autoscaling, and routing.
  • Partner with engineering on release readiness, compatibility, and regression testing so customers can trust new capabilities.

Skills

AI/ML product management
Performance optimization
Cross‑functional leadership
Benchmarking
Customer engagement

Education

Bachelor's degree in CS/EE or related field

Tools

PyTorch
vLLM
TensorRT-LLM
NVIDIA Dynamo

Job description

Baseten is hiring a deeply technical Product Manager to accelerate AI inference performance at scale. You will work alongside inference engineers and researchers to turn model-serving advances into tangible customer features that improve latency, throughput, and cost in real production workloads.

You will bridge cutting-edge systems engineering with real-world workloads, helping customers choose optimal configurations and ensuring smooth release cycles.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Product Manager, AI Inference Platform at Scale
Product Manager, AI Inference Platform at Scale

Black.ai • New York (NY)

On-site
USD 160,000 - 240,000
Equity
Medical, dental, vision insurance (US)
Flexible PTO
+3
Product Manager, AI Inference & Dev Experience
Product Manager, AI Inference & Dev Experience

present of AI • San Francisco (CA)

On-site
USD 140,000 - 200,000
Platform Marketing Manager, AI Inference & Scale
Platform Marketing Manager, AI Inference & Scale

The AI Landscape • New York (NY)

Hybrid
USD 170,000 - 220,000
Equity
Health insurance
Dental and vision coverage
+4
Product Manager, AI Inference Platform
Product Manager, AI Inference Platform

Baseten • San Francisco (CA)

On-site
USD 180,000 - 240,000
Competitive equity & compensation
US medical/dental/vision coverage for
Flexible PTO including Winter Break
+4
Technical Program Manager, AI Inference & Model Performance
Technical Program Manager, AI Inference & Model Performance

Baseten • New York (NY)

On-site
USD 165,000 - 330,000
Equity
Medical, dental, vision insurance
Flexible PTO
+3
Senior Product Manager, AI Inference Platform
Senior Product Manager, AI Inference Platform

AItoolnavio • San Francisco (CA)

On-site
USD 170,000 - 250,000
Competitive compensation
Equity
Medical, dental, vision insurance for
+5
Backend Platform Engineer - AI Inference & Scale
Backend Platform Engineer - AI Inference & Scale

present of AI • San Francisco (CA)

On-site
USD 140,000 - 200,000
Product Manager, Inference Experience
Product Manager, Inference Experience

present of AI • San Francisco (CA)

On-site
USD 140,000 - 200,000
Inference Platform Engineer, Production Tools & APIs
Inference Platform Engineer, Production Tools & APIs

Baseten • Seattle (WA)

On-site
USD 165,000 - 330,000
Equity
Medical/Dental/Vision insurance
Flexible PTO
+4
Technical Product Marketing Lead — AI Inference & Platforms
Technical Product Marketing Lead — AI Inference & Platforms

Baseten • San Francisco (CA)

On-site
USD 140,000 - 190,000
Competitive compensation
Equity
Paid time off
+4