Senior AI Inference Data Plane Engineer

DigitalOcean; Boston (MA); On-site; USD 139,200 - 174,000; Full time

Benefits offered by this job

Equity compensation

Conference reimbursement

Learning stipend

Flexible time off

Employee Assistance Program

Job summary

DigitalOcean is expanding its AI Infrastructure layer and seeks a Senior Engineer 2 to lead high-scale data plane services powering Inference as a Service. You will design, develop, and operate distributed inference systems at scale, collaborating with product, research, and ops teams.

The role emphasizes Go or Python, Kubernetes, and open-source contributions, with opportunities for mentorship and impact across a global remote-first engineering culture.

Qualifications

• Hands-on experience hosting large language or multimodal models using inference engines.
• Experience with distributed inference serving frameworks and cloud-scale systems.
• Strong knowledge of LLM architectures and optimization techniques.
• Proven track record contributing to open source projects.

Responsibilities

• Drive end-to-end design, development, and delivery of data plane components.
• Architect high-scale, multi-tenant AI inference cloud systems.
• Optimize distributed inference hosting and KV-cache routing.
• Collaborate with product managers and cross-functional teams.

Skills

AI/ML Domain Knowledge

Inference Frameworks

Inference Engine Depth

Distributed Inference Fluency

Upstream Track Record

Architecture Proficiency

Software Engineering

Cloud Operations

Open Source Mindset

Tools

Go

Python

gRPC

Job description

DigitalOcean is expanding its AI Infrastructure layer and seeks a Senior Engineer 2 to lead high-scale data plane services powering Inference as a Service. You will design, develop, and operate distributed inference systems at scale, collaborating with product, research, and ops teams.

The role emphasizes Go or Python, Kubernetes, and open-source contributions, with opportunities for mentorship and impact across a global remote-first engineering culture.