Senior Forward Deployed Engineer I (AI Inference)

DigitalOcean; Bengaluru; On-site; INR 4,500,000 - 7,000,000; Full time

Job summary

DigitalOcean in Bengaluru, India is seeking a Senior Forward Deployed Engineer I to own end-to-end LLM deployment for high-growth AI customers. You will embed with founders to solve high-throughput, low-latency inference, architect distributed services, and ship code that improves TTFT and TPOT in production-grade GPU clusters.

You’ll collaborate with customers and internal AI teams, contribute improvements to open-source projects like vLLM and llm-d, and advocate hardware-efficient inference

Qualifications

• 6+ years in AI/ML systems and distributed inference for cluster-scale serving.
• Experience with multi-tenant LLM inference engines and production-grade deployments.
• Strong programming skills in Python or Go and familiarity with Kubernetes.
• Knowledge of GPU-accelerated AI workloads and model serving architectures.

Responsibilities

• Lead end-to-end AI inference design, development, and delivery for high-value workloads.
• Architect and deploy production-grade, multi-tenant LLM inference engines.
• Embed with external customers to debug latency, profile memory, and refactor code for high concurrency.
• Improve cluster-scale serving, including KV-cache locality and prefill/decode strategies.
• Contribute performance fixes to open-source inference ecosystems (vLLM, llm-d).

Skills

AI/ML systems

Distributed systems

Python/Go

Kubernetes

LLM inference

gRPC

Tools

llm-d

vLLM

SGLang

TensorRT-LLM

Job description

Dive in and do the best work of your career at DigitalOcean. Journey alongside a strong community of top talent who are relentless in their drive to build the simplest scalable cloud. If you have a growth mindset, naturally like to think big and bold, and are energized by the fast-paced environment of a true industry disruptor, you’ll find your place here. We value winning together—while learning, having fun, and making a profound difference for the dreamers and builders in the world.

Position Overview

We are looking for a Senior Forward Deployed Engineer I (FDE) who is passionate about operationalizing and collaborating closely with strategic AI enterprises and high-growth startups. As AI-native startups scale, serving large generative AI models at low latency without blowing up infrastructure budgets is their biggest hurdle. That’s where you come in. As a Sr. Forward Deployed AI Inference Engineer (FDE) I, you won't just optimize benchmarks in a lab—you will embed directly with high-growth AI founders to solve high-throughput, cluster-scale LLM serving challenges on DigitalOcean’s GPU cloud. You operate at the intersection of distributed systems, specialized AI hardware, and real-world customer impact. You will own the full lifecycle of high-performance LLM deployment, turning raw GPU compute into hyper-efficient, resilient serving infrastructure for our high-value customers. You will spend your time architecting distributed inference systems, profiling network bottlenecks, debugging KV-cache locality issues, and shipping code that directly reduces time-to-first-token (TTFT) and time-per-output-token (TPOT) for top AI teams.

What You’ll Do

• Technical Leadership: Act as an AI Inference lead on the FDE team, driving the end-to-end design, development, and delivery of critical AI workloads leveraging large generative AI models.
• Design & Scale Distributed AI Inference Systems: Architect and deploy production-grade, multi-tenant LLM inference engines using Kubernetes-native frameworks like llm-d, NVIDIA Dynamo, Ray Serve, vLLM, and SGLang.
• Lead Forward Deployed Engagement: Embed alongside external tech leads to debug latency spikes, profile GPU memory utilization, and refactor inference code for high-concurrency production workloads. You are the bridge between our customers and our internal AI infrastructure teams.
• Optimize at Cluster Scale: Solve the distributed-systems problems unique to LLM serving. You’ll implement strategies for prefill/decode disaggregation, KV-cache-aware routing, tiered prefix caching, and wide expert parallelism for MoE models.
• Drive Hardware Efficiency: Guide customers on compute efficiency utilizing techniques like tensor/data parallelism, continuous batching, and quantization (FP8/FP4) to inflate tokens-per-second per dollar.
• Build Internal Tooling & Upstream Value: Translate customer edge cases into reusable internal blueprints and contribute performance fixes directly back to open-source inference ecosystems (vLLM, llm-d) on behalf of DigitalOcean.

Travel & Collaboration Requirements

Ability to travel up to 30% for customer engagements, strategic workshops, conferences, and internal collaboration. Ability to consistently overlap with North American business hours, including availability until at least noon Eastern Time, to collaborate effectively with customers, Product, Engineering, and go-to-market teams.

What You’ll Add To DigitalOcean

• Deep Distributed Inference Fluency: 6+ years in AI/ML systems, with a deep understanding of why cluster-scale serving is hard (e.g., partitioning KV-cache across workers, fast cross-pod KV transfer, and inference-aware load balancing).
• Framework Mastery: Hands‑on experience with vLLM, llm-d, SGLang, TensorRT-LLM, or Modular MAX, including a solid grasp of internals like continuous batching and paged attention.
• Code & Architecture Proficiency: Expert-level proficiency in Python or GoLang, familiarity with gRPC, and experience running critical services on Kubernetes in high-scale environments.
• Customer-Facing Engineering Mindset: A clear communicator with a growth mindset who enjoys pairing with external engineering teams. You can seamlessly translate business latency SLAs into deep technical infrastructure solutions.
• The DO "Shark" Mentality: You think big, bold, and scrappy. You have a bias for action and a powerful sense of ownership over the customer experience.

Preferred Qualifications

AI Inference & Forward Deployed Engineering Experience: 6+ years of experience working in Forward Deployed Engineering, AI Inference architect, Technical Consulting roles supporting production AI systems.

• Customer Empathy & Technical Leadership: Ability to translate complex business tasks into AI engineering solutions and collaborate directly with client teams (CTOs, AI Leads).
• Builder Mentality: Preference for delivering production-ready code, low-latency container images, and deployment blueprints over slide decks.
• Agility: Comfortable navigating fast-moving environments and tuning model workloads for diverse accelerator architectures.

Vendor & Strategic Partnership Collaboration

• This job is located in Bengaluru, India

Why You’ll Like Working for DigitalOcean

• We innovate with purpose. You’ll be a part of a cutting‑edge technology company with an upward trajectory, who are proud to simplify cloud and AI so builders can spend more time creating software that changes the world. As a member of the team, you will be a Shark who thinks big, bold, and scrappy, like an owner with a bias for action and a powerful sense of responsibility for customers, products, employees, and decisions.
• We prioritize career development. At DO, you’ll do the best work of your career. You will work with some of the smartest and most interesting people in the industry. We are a high-performance organization that will always challenge you to think big. Our organizational development team will provide you with resources to ensure you keep growing. We provide employees with reimbursement for relevant conferences, training, and education. All employees have access to LinkedIn Learning's 10,000+ courses to support their continued growth and development.
• We care about your well‑being. Regardless of your location, we will provide you with a competitive array of benefits to support you from our Employee Assistance Program to Local Employee Meetups to flexible time off policy, to name a few. While the philosophy around our benefits is the same worldwide, specific benefits may vary based on local regulations and preferences.
• We reward our employees. The salary range for this position is based on market data, relevant years of experience, and skills. You may qualify for a bonus in addition to base salary; bonus amounts are determined based on company and individual performance. We also provide equity compensation to eligible employees, including equity grants upon hire and the option to participate in our Employee Stock Purchase Program.
• DigitalOcean is an equal‑opportunity employer. We do not discriminate on the basis of race, religion, color, ancestry, national origin, caste, sex, sexual orientation, gender, gender identity or expression, age, disability, medical condition, pregnancy, genetic makeup, marital status, or military service.