Senior Director, AI Inference & Optimization

DigitalOcean; San Francisco (CA); On-site; USD 274,400 - 343,000; Full time

Job summary

DigitalOcean is seeking a Senior Director of Engineering to lead a high‑performing team building and scaling our LLM inference products across control plane, optimization, and architecture. You will own Serverless Inference, Dedicated Inference, Inference Router, Batch Inference, and Multimodal Inference, driving cost‑efficient systems for millions of users worldwide.

You will lead product strategy, collaborate with PMs and engineers, and champion observability, performance, and reliability

Qualifications

• 10+ years of software engineering experience.
• 6+ years in a technical leadership or management role.
• Experience in Inference Systems or AI/ML systems preferred.

Responsibilities

• Lead and mentor an engineering team focused on LLM inference products.
• Define and execute the product roadmap for DigitalOcean's inference suite.
• Drive the design of the inference serving stack, optimization, and model architecture.
• Collaborate with Product, other engineering teams, and stakeholders on priorities and risks.
• Ensure production health, SLAs, and on-call processes.

Skills

Team Leadership

Distributed systems

Kubernetes at scale

LLM inference

AI workload orchestration

Communication

Ownership

Tools

vLLM

SGLang

LLM-D

NVLink

Job description

DigitalOcean is seeking a Senior Director of Engineering to lead a high‑performing team building and scaling our LLM inference products across control plane, optimization, and architecture. You will own Serverless Inference, Dedicated Inference, Inference Router, Batch Inference, and Multimodal Inference, driving cost‑efficient systems for millions of users worldwide.

You will lead product strategy, collaborate with PMs and engineers, and champion observability, performance, and reliability