Director of Product

FriendliAI; San Francisco (CA); On-site; USD 210,000 - 280,000; Full time

Benefits offered by this job

Flexible working hours

Daily meals provided

Unlimited snacks and beverages

Supportive, collaborative environment

Health check-up support and top-tier设备

Startup equity and competitive pay

Job summary

FriendliAI is seeking a Director of Product to own the strategy and evolution of its AI inference platform. You will define how the platform serves models, scales workloads, and delivers gains in performance, reliability, and cost.

This hands-on leadership role requires deep understanding of AI infrastructure and strong collaboration with engineering and research. You will drive end-to-end product initiatives, mentor PMs/PDs, and establish measurement practices to move quickly while maintaining

Qualifications

• 10+ years of product management experience with technically complex platforms or infrastructure.
• Proven experience leading product strategy for AI/ML infrastructure, cloud platforms, or distributed systems.
• Deep understanding of AI inference and LLM serving architectures.

Responsibilities

• Own the product vision, strategy, and roadmap for the AI inference platform.
• Balance performance, scalability, reliability, usability, and cost across the stack.
• Translate complex capabilities into clear customer value and experiences.
• Lead and mentor PMs and product designers to high standards.

Skills

Product leadership

AI infrastructure

Distributed systems

Strategic thinking

Cross-functional

Technical fluency

Customer focus

Communication

Mentoring

Roadmapping

Job description

About the job

FriendliAI is looking for a Director of Product to own the product strategy and evolution of our AI inference platform. As Director of Product Management, you will define how FriendliAI’s inference platform serves models, scales across demanding workloads, and delivers measurable gains in performance, reliability, and cost.

AI inference is an unusually complex product problem. Models, GPUs, networking, scheduling, serving systems, and developer workflows must work together under real production constraints. Customers expect low latency and high throughput as workloads constantly change, and the platform must make this complexity feel simple to the engineers building on top of it.

This is a hands‑on product leadership role for a technically strong product leader who understands AI infrastructure, works deeply with engineering and research, and translates complex systems into products customers want to use. You will own product direction across the inference platform while helping build and lead the product organization as FriendliAI scales.

Key Responsibilities

Product Strategy & Platform

• Own the product vision, strategy, and roadmap for FriendliAI’s AI inference platform, spanning model serving, deployment, orchestration, APIs, and developer‑facing capabilities.

• Define product direction across the inference stack, balancing performance, scalability, reliability, usability, and cost.

• Identify and prioritize the highest‑impact product opportunities across rapidly evolving AI workloads, including LLMs, multimodal models, and agentic applications.

• Translate complex technical capabilities into clear product value and differentiated customer experiences.

Inference & Customer Experience

• Develop a deep understanding of AI inference workloads and the challenges faced by ML engineers, AI platform teams, and infrastructure organizations.

• Partner closely with engineering and research to shape capabilities across routing, batching, scheduling, resource allocation, model optimization, and inference performance.

• Drive product strategy around critical inference metrics including latency, time‑to‑first‑token, throughput, GPU utilization, cost per inference, and availability.

• Work directly with enterprise customers and technical users to understand production workloads, validate product direction, and identify high‑impact opportunities.

• Partner with Sales, Solutions, Customer Success, and Marketing to translate customer needs into product priorities and strengthen FriendliAI’s market position.

Product Leadership

• Lead product initiatives end‑to‑end, from discovery and problem definition through prioritization, execution, launch, and iteration.

• Lead and mentor Product Managers and Product Designers, establishing clear ownership and a high bar for product thinking and execution.

• Establish product planning, prioritization, and measurement practices that help the organization move quickly while maintaining strong product judgment.

• Define and track product KPIs and use customer feedback, product usage, and performance data to guide investment and prioritization.

• Communicate product vision, priorities, and tradeoffs clearly to engineering, research, GTM, and executive leadership.

Qualifications

• 10+ years of product management experience, with significant experience owning technically complex platforms or infrastructure products.

• Proven experience leading product strategy for AI/ML infrastructure, cloud platforms, distributed systems, developer platforms, or related technical products.

• Deep understanding of AI inference and LLM serving architectures, including concepts such as prefill vs. decode, KV cache, token streaming, batching, routing, and GPU utilization.

• Strong technical fluency and the ability to work directly with senior engineering and research teams on complex distributed systems.

• Proven track record taking technically complex products from strategy through execution and customer adoption.

• Experience leading and developing Product Managers and/or coordinating multiple product workstreams.

• Strong customer orientation, with experience working directly with enterprise customers and technical users.

• Strong written and verbal communication, with the ability to explain complex technical systems and make clear product tradeoffs.

• Strong product judgment and the ability to operate effectively in a fast‑moving environment with significant ambiguity.

Preferred Experience

• Experience with AI inference platforms, LLM serving, GPU infrastructure, or model deployment.

• Familiarity with modern inference frameworks such as vLLM, TensorRT-LLM, SGLang, or similar technologies.

• Experience building developer platforms, APIs, SDKs, or cloud‑native infrastructure products.

• Experience with multi‑tenant SaaS or enterprise AI platforms.

• Experience working with ML engineers, AI platform teams, or infrastructure organizations.

• Background in Computer Science, Engineering, or a closely related technical field.

Benefits

• Flexible working hours

• Daily lunch and dinner provided; unlimited snacks and beverages

• Supportive and highly collaborative work environment

• Health check‑up support and top‑tier equipment/hardware support

• A front‑row seat to the generative AI infrastructure revolution

• Competitive compensation, startup equity, health insurance, and other benefits

About FriendliAI

FriendliAI is the fastest inference cloud for agents, built to run frontier open‑weight models in production at scale. It delivers up to 7x faster output token speed, up to 90% lower inference costs, and 99.99% uptime across the most demanding agent workloads — long‑context inference, real‑time streaming, and accurate tool calling.

We are a small, fast‑moving team doing work that matters at one of the most exciting moments in technology history. With our world‑class inference stack, we are building the platform teams can actually rely on.