Product Manager, AI Inference Platform at Scale

Black.ai

New York (NY)

On-site

USD 160,000 - 240,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Equity
Medical, dental, vision insurance (US)
Flexible PTO
Parental leave
Fertility stipend
401(k)

Job summary

Baseten is building a new cloud for AI, enabling inference, training, and agent execution as a cohesive platform for developers. We seek a deeply technical Product Manager to partner with inference engineers and researchers, turning model-serving advances into features customers can adopt in production.

You will bridge the gap between advanced systems engineering and real-world workloads, helping customers leverage performance improvements at scale and across diverse model configurations.

Qualifications

  • Experience shipping AI/ML products or infrastructure where performance and production behavior mattered.
  • Technical background in software/ML engineering or research.
  • Strong understanding of model serving architectures, GPU economics, latency and cost tradeoffs.
  • Experience partnering with researchers or infra engineers to bring capabilities into production.
  • A rigorous approach to benchmarking and performance measurement.

Responsibilities

  • Work with inference engineering and research teams to bring new performance capabilities to customers.
  • Translate advances in batching, caching, quantization, and serving architectures into product features.
  • Define benchmarks and evaluations showing latency, throughput, quality and cost improvements.
  • Understand customer workloads and prioritize optimizations with greatest impact.
  • Make performance features easier to configure, evaluate, and adopt in production.
  • Help customers understand tradeoffs across hardware, model configs, autoscaling, and routing.
  • Partner with engineering on release readiness, compatibility, and regression testing.

Skills

AI/ML product experience
Performance-focused
Technical software/ML background
Benchmarking & experimentation
Customer-value translation

Tools

PyTorch
TensorRT-LLM
vLLM

Job description

Baseten is building a new cloud for AI, enabling inference, training, and agent execution as a cohesive platform for developers. We seek a deeply technical Product Manager to partner with inference engineers and researchers, turning model-serving advances into features customers can adopt in production.

You will bridge the gap between advanced systems engineering and real-world workloads, helping customers leverage performance improvements at scale and across diverse model configurations.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Product Manager, AI Inference - Scale & Performance
Product Manager, AI Inference - Scale & Performance

AItoolnavio • San Francisco (CA)

On-site
USD 150,000 - 190,000
Equity & competitive compensation
Health insurance (US)
Flexible PTO (Winter Break)
+3
Product Manager, AI Inference Platform
Product Manager, AI Inference Platform

Baseten • San Francisco (CA)

On-site
USD 180,000 - 240,000
Competitive equity & compensation
US medical/dental/vision coverage for
Flexible PTO including Winter Break
+4
Product Manager, AI Inference & Dev Experience
Product Manager, AI Inference & Dev Experience

present of AI • San Francisco (CA)

On-site
USD 140,000 - 200,000
Senior Product Manager, AI Inference Platform
Senior Product Manager, AI Inference Platform

AItoolnavio • San Francisco (CA)

On-site
USD 170,000 - 250,000
Competitive compensation
Equity
Medical, dental, vision insurance for
+5
Platform Marketing Manager, AI Inference & Scale
Platform Marketing Manager, AI Inference & Scale

The AI Landscape • New York (NY)

Hybrid
USD 170,000 - 220,000
Equity
Health insurance
Dental and vision coverage
+4
Product Manager, Inference Experience
Product Manager, Inference Experience

present of AI • San Francisco (CA)

On-site
USD 140,000 - 200,000
Backend Platform Engineer - AI Inference & Scale
Backend Platform Engineer - AI Inference & Scale

present of AI • San Francisco (CA)

On-site
USD 140,000 - 200,000
Senior AI Platform Product Manager
Senior AI Platform Product Manager

Black.ai • San Francisco (CA)

On-site
USD 180,000 - 260,000
Competitive compensation with equity
100% medical, dental, vision for you &
Flexible PTO including Winter Break
+4
Technical Product Marketing Lead, AI Platform
Technical Product Marketing Lead, AI Platform

Baseten • New York (NY)

On-site
USD 180,000 - 220,000
Competitive compensation
Equity
Healthcare (US)
+4
Technical Product Marketing Lead — AI Inference & Platforms
Technical Product Marketing Lead — AI Inference & Platforms

Baseten • San Francisco (CA)

On-site
USD 140,000 - 190,000
Competitive compensation
Equity
Paid time off
+4