Senior LLM Inference Optimization Engineer

Rakuten Asia Pte Ltd; Singapore; Hybrid; SGD 120,000 - 190,000; Full time

Job summary

Rakuten Asia Pte. Ltd. in Singapore is seeking a Senior Software Engineer focusing on LLM inference optimization.

You will profile, optimize, and extend inference engines to improve throughput, reduce latency, and maximize GPU utilization on NVIDIA GPUs. The role requires deep CUDA/Triton programming, experience with vLLM, TensorRT-LLM, or Triton Inference Server, plus strong Python and C++ software engineering.

Qualifications

• 3+ years of hands-on experience optimizing deep learning inference on NVIDIA GPUs.
• Strong knowledge of LLM inference internals: attention mechanisms, KV cache, batching strategies, quantization, and parallelism.
• Working experience with at least one mainstream inference serving engine (vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or equivalent).
• Proficiency in CUDA and/or Triton kernel programming, with a solid understanding of modern GPU architecture.
• Strong software engineering fundamentals in Python and C++.
• Bachelor's or higher degree in Computer Science, Engineering, or a related field.

Responsibilities

• Profile, optimize, and extend modern LLM inference engines (e.g., vLLM, SGLang, TensorRT-LLM) to improve throughput, latency, and GPU utilization.
• Design and implement inference-time optimizations such as quantization, KV-cache management, continuous batching, speculative decoding, and parallelism strategies (tensor / expert / pipeline).
• Develop and tune GPU kernels (CUDA, Triton) for critical operators — attention, GEMM, MoE — targeting the latest NVIDIA architectures.
• Build benchmarking and load-testing frameworks to measure end-to-end serving performance under realistic workloads and SLOs.
• Partner with infrastructure teams to deploy and scale inference on Kubernetes-based GPU clusters, including autoscaling, fault tolerance, and observability.
• Stay current with the fast-moving inference research landscape and bring cutting-edge techniques into production.

Skills

LLM inference optimization

CUDA

Python

C++

Triton

GPU kernels

quantization

KV cache

batching strategies

inference serving

Education

Bachelor's or higher in CS/Engineering

Tools

vLLM

SGLang

TensorRT-LLM

Triton Inference Server

CUDA

Job description

Rakuten Asia Pte. Ltd. in Singapore is seeking a Senior Software Engineer focusing on LLM inference optimization.

You will profile, optimize, and extend inference engines to improve throughput, reduce latency, and maximize GPU utilization on NVIDIA GPUs. The role requires deep CUDA/Triton programming, experience with vLLM, TensorRT-LLM, or Triton Inference Server, plus strong Python and C++ software engineering.