Senior AI Infrastructure & Tooling TPM

Deepgram; San Francisco (CA); On-site; USD 210,000 - 270,000; Full time

Job summary

Deepgram in San Francisco is seeking a Senior Technical Program Manager (AI Tooling & Systems) to drive end-to-end ML infrastructure programs, spanning model training pipelines, inference serving, and internal tooling.

You'll collaborate with ML researchers, engineers, and product teams to optimize cost, latency, and scalability, build scalable processes, and translate cutting-edge research into production-ready systems.

Qualifications

• 5+ years of program management or technical leadership in ML infrastructure.
• Strong technical acumen in ML systems; hands-on ML engineer or systems engineer experience.
• Experience coordinating cross-functional ML programs across training, evaluation, serving, monitoring.
• Proven ability to translate ML requirements into scalable infrastructure.
• Comfortable with ambiguity and complex tradeoffs (accuracy vs latency vs cost).
• Excellent communication with technical and non-technical stakeholders.
• Familiarity with high-growth or startup environments.

Responsibilities

• Own end-to-end delivery of AI infrastructure programs, including training pipelines and inference serving.
• Define architecture, integration patterns, and rollout strategies for ML systems and tooling.
• Bridge ML research, ML engineering, product, and data teams to align roadmaps and timelines.
• Drive cost and latency optimization for real-time inference at scale.
• Build internal tools to accelerate ML iteration (tracking, versioning, A/B testing).
• Identify and resolve bottlenecks in training, serving, and evaluation workflows.
• Collaborate with ML practitioners to translate research into scalable, observable systems.

Skills

ML infrastructure

Program management

Cross-functional collaboration

Technical leadership

Tools

PyTorch

TensorRT

TorchServe

MLflow

Weights & Biases

DVC

vLLM

CUDA

Job description

Deepgram in San Francisco is seeking a Senior Technical Program Manager (AI Tooling & Systems) to drive end-to-end ML infrastructure programs, spanning model training pipelines, inference serving, and internal tooling.

You'll collaborate with ML researchers, engineers, and product teams to optimize cost, latency, and scalability, build scalable processes, and translate cutting-edge research into production-ready systems.