AI Platform Engineer (Inference)

Amazon; Bengaluru; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Amazon Bengaluru seeks an experienced backend and distributed-systems engineer to scale AI inference platforms. You will use Python or Go to design reliable services and help deploy GPU-accelerated workloads.

Ideal candidates will know inference frameworks (vLLM, Triton, TensorRT-LLM, ONNX Runtime, PyTorch), distributed training concepts, and cloud-native tools like Kubernetes and container orchestration. Join a team driving high-performance ML at scale.

Qualifications

• 9+ years of backend development / distributed systems experience with Python or Go.
• Hands-on with inference frameworks: vLLM, Triton, TensorRT-LLM, ONNX Runtime, PyTorch.
• Understanding of distributed training: data parallelism, tensor parallelism, pipeline parallelism, ZeRO.
• Knowledge of Kubernetes and cloud-native ecosystem; containerization and service orchestration.
• Proficiency in GPU cluster management fundamentals (CUDA, cuDNN, device plugins).
• Strong distributed systems design: load balancing, HA, resource scheduling.

Skills

Backend development

Distributed systems

Python

Go

Inference frameworks

Kubernetes

Cloud-native

GPU cluster management

Distributed training

Load balancing

Tools

vLLM

Triton

TensorRT-LLM

ONNX Runtime

PyTorch

CUDA

cuDNN

Docker

Job description

Requirements:

• 9+ years of backend development / distributed systems; Python or Go.
• Hands-on experience with one or more inference frameworks: vLLM, Triton, TensorRT-LLM, ONNX Runtime, PyTorch.
• Understanding of distributed training: data parallelism, tensor parallelism, pipeline parallelism, ZeRO.
• Knowledge of Kubernetes and cloud-native ecosystem; containerization and service orchestration.
• Proficiency in GPU cluster management fundamentals (CUDA, cuDNN, device plugins).
• Strong distributed systems design: load balancing, HA, resource scheduling.

Nice-to-Have:

• LLM serving at scale (vLLM, SGLang, TensorRT-LLM) in production.
• Quantisation techniques: INT8/FP8 sparsity, speculative decoding.
• Large-scale GPU training cluster operation (Slurm, Volcano, KubeRay).
• Training fault recovery, checkpoint management, elastic training.
• RDMA/NCCL/InfiniBand high-performance networking.
• Active open-source contributions (vLLM, SGLang, Megatron-LM, DeepSpeed, Ray).