AI Platform PM: Orchestration, Eval & Safe Deployment

RiverPark Ventures; New York (NY); Hybrid; USD 190,000 - 220,000; Full time

Benefits offered by this job

Flexible PTO

12 paid holidays + DisConnect Days

Lifestyle stipend

Disco credit

Childcare stipend

Commuter benefit

Home office setup reimbursement

Grubhub credit

401(k)

Comprehensive insurance

Job summary

Disco is hiring an AI/ML Product Manager to own the end-to-end AI platform, including model orchestration, routing, and eval loops. You’ll ensure model-agnostic, harness-agnostic systems and manage cost considerations as a strategic business issue.

You’ll design, test, and ship production-grade AI workflows with a small, fast team. You’ll work across reliability, data, and engineering to rewrite the orchestration, compare models, and keep governance and safety top of mind while enabling rapid

Qualifications

• Shipped real AI products or platforms into production.
• Able to reason about model tradeoffs, latency, and cost.
• Able to redesign orchestration and evaluate models/harnesses against each other.

Responsibilities

• Own the AI platform orchestration, routes, and evaluation loops.
• Build and own production safety harnesses and verification processes.
• Productize internal AI tooling and automate workflows and memory/context management.
• Set standards for trusted AI outputs and verification before shipping.
• Explain model routing decisions clearly to leadership and engineers.

Skills

Product management

Systems thinking

Model routing

Cost awareness

Data analysis

Technical fluency

Vendor-agnostic

Tools

AWS Bedrock

AI tooling

Job description

Disco is hiring an AI/ML Product Manager to own the end-to-end AI platform, including model orchestration, routing, and eval loops. You’ll ensure model-agnostic, harness-agnostic systems and manage cost considerations as a strategic business issue.

You’ll design, test, and ship production-grade AI workflows with a small, fast team. You’ll work across reliability, data, and engineering to rewrite the orchestration, compare models, and keep governance and safety top of mind while enabling rapid