Principal Engineer: AI Inference Memory & Storage

DigitalOcean; Seattle (WA); Híbrido; USD 250.000 - 312.000; Jornada completa

Descripción de la vacante

DigitalOcean is seeking a Principal Engineer to own the memory and storage layer for the inference platform. You will drive a unified, multi-tier memory substrate spanning GPU HBM, host DRAM, NVMe, and remote storage, and integrate with serving engines like vLLM and TensorRT-LLM.

You’ll lead cross-team architecture, contribute to open source projects, and shape unit economics for cost and performance at scale.

Formación

• 15+ years building large-scale distributed systems or ML infrastructure.
• Fluency in memory hierarchies and multi-tier storage design.
• Strong systems programming skills in C/C++, Go, Rust, or Python.

Responsabilidades

• Define and evolve a unified memory layer across GPU memory, host memory, NVMe, and remote object storage.
• Architect deep integrations with serving engines (vLLM, SGLang, TensorRT-LLM) for KV cache offload and routing.
• Design interfaces for disaggregated prefill/decode and cache transfer across nodes.
• Own eviction/admission strategies and metrics for TTFT, ITL, and throughput.
• Collaborate with GPU infrastructure, networking, and platform teams to leverage GPUDirect, RDMA, and NVMe-oF.
• Model and validate impact of cache hit ratio and tiering on cost per token and per GPU-hour.
• Set technical direction, mentor engineers, and represent DigitalOcean externally.

Conocimientos

Distributed systems experience

Systems programming

Languages: C/C++/Go/Rust/Python

Profiling and optimization

Cross-functional communication

Educación

15+ years in relevant field

Herramientas

RDMA/NVMe-oF

GPUDirect

NVLink

Descripción del empleo

DigitalOcean is seeking a Principal Engineer to own the memory and storage layer for the inference platform. You will drive a unified, multi-tier memory substrate spanning GPU HBM, host DRAM, NVMe, and remote storage, and integrate with serving engines like vLLM and TensorRT-LLM.

You’ll lead cross-team architecture, contribute to open source projects, and shape unit economics for cost and performance at scale.