AI Platform PM: Model Orchestration & Evaluation

Disco Network; New York (NY); Hybrid; USD 190,000 - 220,000; Full time

Benefits offered by this job

Flexible PTO

12 paid holidays

DisConnect Days

Lifestyle stipend

Disco credit

Childcare stipend

Home office setup

Grubhub credit

401(k)

Job summary

Disco is hiring an AI/ML Product Manager to own Disco's AI platform end to end. You’ll ensure model-agnostic and harness-agnostic design, swap in better models rapidly, and own eval loops, guardrails, and the agentic pipeline for a team scaling from 25 to 200+.

This is a product role with deep technical depth requiring a systems mindset and strong product judgment to ship reliable AI infrastructure.

Qualifications

• Shipped real AI products or platforms into production and can point to the eval loops, harnesses, or orchestration built around them.
• Thinks in systems: routing, evaluation, feedback loops, cost, and safety considerations.

Responsibilities

• Own the AI platform end-to-end, including routing tasks to models, evaluating models and harnesses, and refining the orchestration layer.
• Own the cost, safety reviews, and production readiness of AI components and pipelines.
• Design and maintain eval graphs to catch regressions and guardrails to prevent incidents.
• Read code and data to understand workflows, token usage, and performance issues.

Skills

Routing

Eval loops

Harnesses

Orchestration

Model-agnostic

Cost awareness

Product judgment

Cross-team communication

Job description

Disco is hiring an AI/ML Product Manager to own Disco's AI platform end to end. You’ll ensure model-agnostic and harness-agnostic design, swap in better models rapidly, and own eval loops, guardrails, and the agentic pipeline for a team scaling from 25 to 200+.

This is a product role with deep technical depth requiring a systems mindset and strong product judgment to ship reliable AI infrastructure.