Platform Engineer - AI Infrastructure

Sarvam; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Benefits offered by this job

Competitive salary

Health benefits

Flexible working hours

Job summary

Sarvam is seeking a Platform Engineer for its AI Infrastructure team in Bengaluru. This role involves building scalable AI infrastructure, focusing on scheduling, autoscaling, and multi-tenancy for GPU workloads.

The ideal candidate has 5+ years of experience and is proficient in Go or Python, with strong Kubernetes skills. If you're passionate about creating robust systems and want to help build India's sovereign AI platform, this role is for you.

Qualifications

• 5+ years building infrastructure or platform software.
• Strong software engineering in Go or Python.
• Kubernetes expertise at controller level.
• Understanding GPU-specific constraints.
• A product mindset toward internal users.

Responsibilities

• Build scalable serving platforms for machine learning models.
• Implement autoscaling and elasticity layers.
• Design scheduling and orchestration systems.
• Create multi-tenancy and RBAC management.
• Develop observability and cost tooling.

Skills

Kubernetes expertise

Proficient in Go

Proficient in Python

GPU awareness

Software engineering best practices

Tools

Terraform

Crossplane

Kueue

Slurm

Job description

Platform Engineer - AI Infrastructure

About Sarvam

Sarvam is building the bedrock of Sovereign AI for India. The company is developing India’s full-stack sovereign AI platform, building across research, models, infrastructure and applications with a singular focus on making AI genuinely work for India. Sarvam works with leading enterprises and public institutions and is backed by Lightspeed, Peak XV, and Khosla Ventures. Sarvam partners with India’s leading brands, including Tata Capital, SBI Life, CRED, IDFC, and LIC.

About the Role

Sarvam runs a large, multi-vendor GPU fleet that serves two demanding workloads on the same physical infrastructure: training jobs that span hundreds of GPUs, and inference services that must hold a flat p99 under production load. This role builds the platform that sits on top of that fleet - the scheduling, scaling, multi-tenancy, serving, and self‑service layers that let our ML teams use thousands of GPUs without a human in the loop for every job.

This is the build side of infrastructure, not the operate side. Our SREs keep the fleet reliable and carry the pager; you build the systems that make the fleet usable - and that make it break less in the first place. The work is heavy software engineering: you will design and ship control‑plane services, scheduler integrations, autoscaling controllers, the inference‑serving platform, RBAC and quota systems, observability and cost tooling, and the CLI and APIs ML engineers touch every day. You will treat the platform as a product and its internal users as customers.

You should be a strong systems engineer who is fluent in Kubernetes at the controller and internals level, comfortable in Go or Python, and literate in the GPU‑specific constraints - MIG and GPU sharing, gang scheduling, topology‑aware placement, RDMA - that make this different from a generic cloud platform.

What You’ll Do

• The serving platform. The control plane that turns a model artifact into a scalable, multi‑tenant endpoint: intelligent routing and load balancing across replicas, rollout machinery (canary, blue‑green, rollback), traffic splitting, model‑version integration etc.

• The scaling and elasticity layer. Autoscaling for both training (elastic and gang scaling, scale‑to‑fit) and serving (queue‑depth and utilization‑driven), capacity pooling across clusters, burst handling, preemption and reclaim, and efficient bin‑packing across GPUs etc.

• Scheduling and orchestration. The scheduler layer itself - Kueue, Volcano, Slurm‑on‑Kubernetes, or custom controllers - with gang scheduling, priority and preemption, queue fairness, quota enforcement, and topology‑aware placement that keeps a job’s ranks on the same fabric island.

• Multi‑tenancy, RBAC, and isolation. The tenant model and namespacing, RBAC and policy, quota and fair‑share enforcement, MIG / MPS / time‑slicing exposed as self‑service tiers, secrets and credential management, and audit logging.

• Networking components. Platform‑level network plumbing - CNI configuration and custom components, ingress and service routing, RDMA / SR‑IOV exposure into pods, tenant network policy and segmentation, and multi‑cluster connectivity.

• Observability and cost tooling. The metrics, logging, and tracing pipeline as a platform feature with cardinality managed at fleet scale.

• The storage and data path. Abstractions over the parallel filesystem, caching tiers, volume provisioning, and data‑locality‑aware placement.

• Developer experience and self‑service. The layer ML engineers actually touch - CLI, SDK, and APIs, job‑submission abstractions and inner‑loop tooling.

• Provisioning and infrastructure‑as‑code. The control plane that stands clusters up reproducibly - Terraform, Crossplane, or operators; node lifecycle and image management; driver and firmware rollout automation; and multi‑vendor cluster bring‑up.

What We're Looking For

• 5+ years building infrastructure or platform software, with a track record of systems you designed and shipped - not scripts and glue, but services and control planes that others built on.

• Strong software engineering in Go or Python, with the judgment to build maintainable systems and the depth to debug them in production.

• Kubernetes at the controller and internals level - you have written operators or controllers, understand the scheduler and API machinery, and know where its abstractions leak.

• Working literacy in the GPU‑specific platform constraints: MIG and GPU sharing, gang scheduling, topology‑ and fabric‑aware placement, and why training and serving contend for the same hardware.

• A product mindset toward internal users - you design APIs and abstractions people want to use, and you measure the platform by adoption and self‑service, not by tickets closed.

• The range to own a capability end to end, from design through rollout to the documentation that makes it self‑serve.

Bonus Points

• Having built a serving or inference or training platform - routing, autoscaling, and rollout for model endpoints at scale, fine‑tuning as a service etc.

• Experience with GPU scheduling systems (Kueue, Volcano, Slurm, Run:ai, or custom schedulers) in multi‑tenant production.

• Multi‑tenant isolation (MIG, MPS, time‑slicing) shipped as a self‑service platform capability rather than a one‑off configuration.

• Deep Kubernetes networking - CNI internals, custom network components, or RDMA / SR‑IOV in pods.

• On‑premise GPU platform work, including multi‑vendor or Indian NCP environments.

• Open‑source contributions to Kubernetes, scheduling, or GPU‑platform projects.