LLM Inference Optimization Engineer (GPU-Scale)

Rakuten; Singapore; On-site; SGD 150,000 - 230,000; Full time

Job summary

Rakuten Asia Pte. Ltd. in Singapore is seeking an LLM Inference Optimization Engineer to maximize performance and efficiency of large-scale inference on GPU clusters. You will optimize engines like vLLM, TensorRT-LLM, and Triton, and tune CUDA kernels to meet demanding SLOs.

You will collaborate with infrastructure teams to deploy on Kubernetes, implement quantization, KV-cache management, and batching strategies, and stay current with AI inference research to push production performance.

Qualifications

• 3+ years of hands-on experience optimizing deep learning inference on NVIDIA GPUs, preferably for large language models.
• Strong knowledge of LLM inference internals: attention mechanisms, KV cache, batching strategies, quantization, and parallelism.
• Experience with at least one mainstream inference serving engine (vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or equivalent).
• Proficiency in CUDA and/or Triton kernel programming, with a solid understanding of modern GPU architecture.
• Strong software engineering fundamentals in Python and C++.

Responsibilities

• Profile, optimize, and extend modernLLM inference engines to improve throughput, latency, and GPU utilization.
• Design and implement inference-time optimizations such as quantization, KV-cache management, continuous batching, speculative decoding, and parallelism strategies (tensor / expert / pipeline).
• Develop and tune GPU kernels (CUDA, Triton) for critical operators - attention, GEMM, MoE - targeting the latest NVIDIA architectures.
• Build benchmarking and load-testing frameworks to measure end-to-end serving performance under realistic workloads and SLOs.
• Partner with infrastructure teams to deploy and scale inference on Kubernetes-based GPU clusters, including autoscaling, fault tolerance, and observability.
• Stay current with the fast-moving inference research landscape and bring cutting-edge techniques into production.

Skills

GPU optimization

Deep learning inference

Python

C++

NVIDIA GPUs

LLM inference internals

Education

Bachelor’s or higher degree in Computer Science, Engineering, or a related field

Tools

CUDA

Triton

vLLM

SGLang

TensorRT-LLM

Triton Inference Server

Kubernetes

Job description

Rakuten Asia Pte. Ltd. in Singapore is seeking an LLM Inference Optimization Engineer to maximize performance and efficiency of large-scale inference on GPU clusters. You will optimize engines like vLLM, TensorRT-LLM, and Triton, and tune CUDA kernels to meet demanding SLOs.

You will collaborate with infrastructure teams to deploy on Kubernetes, implement quantization, KV-cache management, and batching strategies, and stay current with AI inference research to push production performance.