Senior AI Infra & Tooling TPM – Real‑Time ML at Scale

Deepgram; United States; On-site; USD 150,000 - 230,000; Full time

Job summary

Deepgram is seeking a Senior Technical Program Manager (AI Tooling & Systems) to drive large-scale ML infrastructure initiatives. You will own end-to-end programs spanning model training pipelines, inference serving, and real-time monitoring, partnering with ML researchers, engineers, and product teams to unlock scalable capabilities.

You’ll thrive if you enjoy clarifying complex ML tradeoffs, building tooling to accelerate iteration, and translating research breakthroughs into observable

Qualifications

• 5+ years of program management or technical leadership in ML infrastructure, ML platforms, or AI tooling (or equivalent).
• Hands-on ML systems experience preferred (ML engineer, systems engineer, or ML infra engineer).
• Experience coordinating cross-functional ML programs (model training → evaluation → serving → monitoring).
• Ability to translate ML/research requirements into robust, scalable infrastructure.
• Comfortable with ambiguity and technical tradeoffs (accuracy vs. latency vs. cost).
• Excellent communication with technical and non-technical stakeholders.
• Familiarity with high-growth or startup environments.

Responsibilities

• Own end-to-end delivery of AI infrastructure programs—from model training pipelines to inference serving and production monitoring.
• Define technical architecture, integration patterns, and rollout strategies for new ML systems and tooling.
• Serve as connective tissue between ML research, ML engineering, product, and data teams to align on requirements and timelines.
• Drive cost and latency optimization for real-time inference workloads at scale.
• Build lightweight internal tools and processes to accelerate ML iteration cycles.
• Identify and resolve bottlenecks in training pipelines, serving infrastructure, and evaluation workflows.
• Work closely with ML practitioners to translate research breakthroughs into scalable, observable systems.

Skills

ML infrastructure

Cross-functional leadership

Communication skills

Ambiguity navigation

Tools

vLLM

TensorRT

TorchServe

Job description

Deepgram is seeking a Senior Technical Program Manager (AI Tooling & Systems) to drive large-scale ML infrastructure initiatives. You will own end-to-end programs spanning model training pipelines, inference serving, and real-time monitoring, partnering with ML researchers, engineers, and product teams to unlock scalable capabilities.

You’ll thrive if you enjoy clarifying complex ML tradeoffs, building tooling to accelerate iteration, and translating research breakthroughs into observable