Principal Product Manager, AI Infrastructure and Orchestration

DataRobot, Inc.; Washington; Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Medical, Dental & Vision Insurance

Flexible Time Off

Paid Holidays

Parental Leave

Job summary

DataRobot, Inc. is seeking a senior product leader to own the workload and agent runtime orchestration across Kubernetes-based deployments and accelerators. You will shape the placement, lifecycle, scaling, isolation, and governance of model and agent workloads, partnering with two engineering pods to drive platform evolution.

The role requires deep API thinking, multi-tenancy expertise, and strong technical writing to articulate contracts and roadmaps for scalable AI workloads.

Qualifications

• 6+ years in product management for infrastructure or cloud services.
• 9+ years for principal candidates with a platform layer background.
• Deep technical understanding of GPU and accelerator behavior: topology-aware placement and resource sharing.
• Strong API product judgment and experience drafting API contracts.
• Experience with multi-tenancy, isolation, quotas, and fairness in architectures.
• BS or MS in Computer Science or equivalent hands-on infra experience.

Responsibilities

• Own the workload and agent runtime orchestration: placement, lifecycle, scaling, and isolation on Kubernetes and accelerators.
• Define the API contracts, versioning strategy, and governance for deployment and usage.
• Manage resource placement, capacity planning, and cross-tenant quota controls.
• Collaborate with two engineering pods and align with product teams building on the platform.
• Engage in prototyping, documentation, and roadmap decisions to guide platform evolution.

Skills

Product management

Kubernetes

API design

Platform engineering

Multi-tenancy

Technical writing

Education

BS or MS in Computer Science

Tools

Kubernetes

Docker

Job description

DataRobot delivers AI that maximizes impact and minimizes business risk. Our platform and applications integrate into core business processes so teams can develop, deliver, and govern AI at scale. DataRobot empowers practitioners to deliver predictive and generative AI, and enables leaders to secure their AI assets. Organizations worldwide rely on DataRobot for AI that makes sense for their business — today and in the future. We run agents and models in production for enterprises that cannot move their workloads to a public cloud: regulated industries, sovereign deployments, air‑gapped and customer‑managed clusters. The control plane is the layer that makes that possible. It deploys the agents, it deploys the models those agents call, and it decides how every workload is placed, scaled, isolated, routed to, and torn down across Kubernetes clusters and heterogeneous accelerators, on our cloud and on the customer’s. Agents and models are one deployment problem here. An agent is a long‑lived workload with session state and unpredictable fan‑out, calling models with wildly uneven cost profiles, and both land on the same finite pool of accelerators. Allocating that pool correctly is the job. You will own this layer as a product. You will spend your time in design reviews, in the API contract, and in production data.

What you will own

The deployment and workload API.

Resource model, lifecycle semantics, versioning, backward compatibility, and the error behavior customers integrate against.

Placement and capacity.

How workloads land on nodes and accelerators, how quota and priority work across tenants, and what happens under contention.

Scaling.

Autoscaling signals, cold start and scale‑to‑zero economics, headroom policy, and the cost‑versus‑latency trade‑off as a customer‑facing control.

Agent runtime.

Where an agent runs and for how long, how it is isolated, how its tool calls execute, and how its state survives a restart or an eviction.

Traffic and connectivity.

Ingress and routing for model and agent endpoints, request‑aware load balancing, tenancy boundaries, and private connectivity into customer networks.

Governance and audit.

Who deployed what, who invoked it, under which policy and access control, and whether that record survives a customer audit.

Metering and packaging.

How inference is measured, quota'd, attributed to a tenant, and priced.

Reliability.

The SLOs, the error budget, and the operational surface a platform engineer uses to diagnose a degraded deployment without opening a ticket with us.

You will hold the roadmap for this layer across two engineering pods and align with the product teams building on top of it.

What we are looking for

6+ years in product management for infrastructure, developer platforms, or cloud services, at least 3 of them on Kubernetes‑based or distributed systems products.

Principal candidates bring 9+ years and a platform layer that other product teams built on.

Deep technical understanding of GPU and accelerator behavior: topology‑aware placement, fractional and time‑sliced sharing, MIG, device plugins and the driver and container runtime plumbing underneath them, memory as the binding constraint, and what utilization costs when a tenant holds a GPU it is not saturating.

Deep technical understanding of Kubernetes: the API server and scheduler, controllers and CRDs, operators, admission and RBAC, device plugins, resource requests and limits, node pools, and what happens when a pod cannot be scheduled.

Multi‑tenancy experience: isolation models, noisy neighbors, quota and fairness, and tenancy designs that survive a customer security review.

API product judgment.

You have owned a public or platform API and lived with the consequences of a contract you shipped. Technical writing and prototyping as your default way to make a case: a doc an engineer will read, a deep dive, a public post, an API reference, or a working prototype.

Comfort operating with matrixed engineering teams and no direct reports.

BS or MS in Computer Science or a closely related technical field, or equivalent hands‑on experience as a software, platform, or infrastructure engineer.

Nice to have

Service networking depth: ingress and routing, load balancing under uneven request cost, DNS, TLS termination, private link connectivity, and network policy as a tenancy boundary.

Modern serving stacks and their failure modes: vLLM or similar, KV cache behavior, batching, quantization trade‑offs.

Long‑running and agentic workload patterns: session affinity, statefulness, tool‑call fan‑out, sandboxed execution.

Customer‑managed, air‑gapped, or sovereign deployments and the compliance constraints that come with them.

Regulated‑industry experience where access control, secrets, and invocation‑level auditability are product requirements.

CNCF or open‑source contribution.

How we work with AI

We use AI tooling in the loop daily and expect the same here. The strongest candidates build to think: a prototype that exercises an API before it is specified, an evaluation harness, an agent wired against real services, a throwaway tool that answers a roadmap question in an afternoon. Bring one or two things you built yourself. Production‑grade code is not the bar; getting to a first version without waiting on an engineer is.

Scope of the role

This role owns workload and agent runtime orchestration: placement, lifecycle, scaling, and isolation of agent and model workloads on Kubernetes and accelerators. It does not own agent coordination and authoring (planner and router logic, memory strategy, multi‑agent handoff, builder surfaces), workflow orchestration in the Airflow or Temporal sense, model quality and applied research, or cluster fabric work such as InfiniBand and RoCE tuning for distributed training. There are no direct reports today. Scope here is technical surface area and influence across engineering pods.

The talent and dedication of our employees are at the core of DataRobot’s journey to be an iconic company. We strive to attract and retain the best talent by providing competitive pay and benefits with our employees’ well‑being at the core.

Here’s what your benefits package may include depending on your location and local legal requirements:

Medical, Dental & Vision Insurance, Flexible Time Off Program, Paid Holidays, Paid Parental Leave, Global Employee Assistance Program (EAP) and more!

DataRobot Operating Principles:

Wow. Our Customers Set High Standards. Be Better Than Yesterday. Be Rigorous. Assume Positive Intent. Have the Tough Conversations. Be Better Together. Debate, Decide, Commit, Deliver Results. Overcommunicate. Research shows that many women only apply to jobs when they meet 100% of the qualifications while many men apply to jobs when they meet 60%.

At DataRobot we encourage ALL candidates, especially women, people of color, LGBTQ+ identifying people, differently abled, and other people from marginalized groups to apply to our jobs, even if you do not check every box. We’d love to have a conversation with you and see if you might be a great fit.

DataRobot is proud to be an Equal Employment Opportunity and affirmative action employer. We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. DataRobot is committed to working with and providing reasonable accommodations to applicants with physical and mental disabilities. Please see the United States Department of Labor’s EEO poster and EEO poster supplement for additional information.

Use of Artificial Intelligence in Our Hiring Process. DataRobot uses approved AI‑powered tools to support the hiring process in selected regions. These tools may assist in writing job descriptions, reviewing applications, assessing qualifications, and evaluating candidate materials. All decisions regarding applications are made by members of the DataRobot team. All applicant data submitted is handled in accordance with our Applicant Privacy Policy.

DataRobot delivers AI that maximizes impact and minimizes business risk. Our AI applications and platform integrate into core business processes so teams can develop, deliver, and govern AI at scale. DataRobot empowers practitioners to deliver predictive and generative AI, and enables leaders to secure their AI assets. Organizations worldwide rely on DataRobot for AI that makes sense for their business — today and in the future.

For more information, visit our website and connect with us on LinkedIn.

DataRobot has become aware of scams involving false offers of DataRobot employment. The scams and false offers use imposter websites, email addresses, text messages, and other fraudulent means. None of these offers are legitimate, and DataRobot’s recruiting process never involves conducting interviews via instant messages, nor requires candidates to purchase products or services, or to process payments on our behalf. Please note that DataRobot does not ask for money in its recruitment process. DataRobot is committed to providing a safe and secure environment for all job applicants. We encourage all job seekers to be vigilant and protect themselves against recruitment scams by verifying the legitimacy of any job offer before providing personal information or paying any fees. Communication from our company will be sent from a verified email address using [e-mail] email domain. If you receive any suspicious emails or messages claiming to be from DataRobot, please do not respond.

Thank you for your interest in DataRobot, and we look forward to receiving your application through our official channels.

Don’t see the dream job you are looking for? Drop off your contact information and resume and we will reach out to you if we find the perfect fit!