Product Manager, Model Gateway

Mercor

San Francisco (CA)

On-site

USD 140,000 - 230,000

Full time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Benefits offered by this job

Bi-annual performance bonus

Equity grant vesting over 4 years

Relocation bonus up to $15k

Housing bonus $10K within 0.5 miles of

$10K housing bonus

Monthly meals stipend $1.5K

Equinox membership

Laundry reimbursement $200 monthly

Wellness reimbursement $200 monthly

Health, Dental, Vision insurance

Job summary

Mercor in San Francisco is seeking a Platform Product Manager to own the LLM gateway and shared infrastructure. You’ll drive roadmap, policy, and lifecycle decisions affecting all product surfaces using frontier models.

This is a high-leverage role partnering with the infra team and leadership to translate tradeoffs into durable systems. You’ll work with internal PMs and engineers to ensure reliability, cost transparency, and scalable usage policies across surfaces and models, with strong

Qualifications

5+ years as a product manager, focus on platform or developer-facing products.
Background in SWE/ML/DS preferred.
Strong systems thinking for queues, rate limits, and tradeoffs.
High ownership and accountability in ambiguous contexts.
Technical fluency: reading usage data, dashboards, and small PRs.

Responsibilities

Own the LLM gateway end-to-end: roadmap, policy, lifecycle.
Design priority and fairness for queueing and rate-limiting.
Make cost visible and attributable to business health.
Negotiate rate limits and usage policy.
Partner with infrastructure to translate product needs into platform constraints.
Work backwards from business to prioritize gateway decisions.

Skills

Platform product

Developer-facing

Systems thinking

Ownership

Independence

Technical fluency

Stakeholder alignment

Cost & metrics

Tools

SQL

Dashboarding

Job description

About Mercor

Mercor's mission is to organize human intelligence to power the AI economy. We're a leading AI data company, building the layer between human expertise and frontier models. Millions of domain experts on the platform are paid over $4 million per day to train frontier AI models. Mercor's APEX benchmark family measures AI's real-world impact on professional work. Mercor Enterprise brings this same infrastructure to Fortune 500 companies: helping companies capture how their best people actually work, translating that expertise directly back into agents.

Mercor is creating a new category of work where expertise powers AI advancement. Achieving this requires an ambitious, fast-paced and deeply committed team. You’ll work alongside researchers, operators, and AI companies at the forefront of shaping the systems that are redefining society. Mercor is a profitable Series C company valued at $10 billion. We work in-person five days a week in our San Francisco, NYC, or London offices.

About the Role

In this Platform Product Manager, you'll own Mercor's LLM gateway, the shared layer every product surface depends on to reach frontier models reliably, affordably, and within policy. This is a high-leverage platform role: the decisions you make about priority queueing, cost attribution, rate limits, and model usage policy ripple across every team that builds on top of us. You'll own this product area with significant autonomy, working directly with an infrastructure team that owns the underlying shared services and with leadership to turn ambiguous, cross-cutting tradeoffs into durable systems and clear policy.

Because the gateway sits between every product team and the models they rely on, your users are internal: the PMs and engineers building on the platform, plus the infra and finance partners who care about reliability and spend. Success looks like product teams moving fast without stepping on each other, model costs that are transparent and attributable down to the team and workload, and usage policies that scale as we add surfaces and models.

What You’ll Do

Own the LLM gateway end-to-end: Set the roadmap and policy for the shared layer that routes every product surface to the models it depends on, from priority queueing to cost attribution to model access, and own its lifecycle from conception through execution.
Design priority and fairness: Define the queueing and rate-limiting logic that decides whose workload runs when, balancing revenue blocking traffic against background jobs, and making the tradeoffs explicit and consistent.
Make cost visible and attributable: Work backwards from business outcomes to make product decisions that impact business health and growth.
Negotiate rate limits and usage policy: Turn contention over shared capacity into clear, defensible rules people can plan around.
Partner with infrastructure: Work hand-in-hand with the central infra team translating product needs into platform requirements and platform constraints back into product tradeoffs.
Work backwards from the business: Connect gateway decisions to business health, cost, reliability, and velocity, and use that to prioritize.

What We’re Looking For

Experience: 5+ years as a product manager, ideally on platform, infrastructure, or developer-facing products; prior background in SWE, ML, or DS strongly preferred.
Systems thinking: You reason fluently about queues, rate limits, quotas, and the tradeoffs between latency, throughput, and cost, and you can turn those tradeoffs into policy.
High ownership: You take full accountability for outcomes not just outputs.
Independence: Able to self-direct in ambiguous, cross-cutting contexts.
Technical fluency: Comfortable reading usage data, writing queries, and building scrappy dashboards or making small PRs to validate direction and unblock engineering.
Stakeholder alignment: You can bring many product teams around shared constraints and turn contention over capacity and spend into rules people accept and plan around.
Cost & metrics orientation: Comfortable with cost models, unit economics, and attribution, and equally able to push forward a longer-term platform vision.