GPU-Accelerated LLM Inference Engineer

Rakuten Asia Pte Ltd; Singapore; On-site; SGD 140,000 - 200,000; Full time

Job summary

Rakuten Asia Pte. Ltd. in Singapore is seeking an LLM Inference Optimization Engineer to maximize the performance, efficiency, and scalability of large-scale inference workloads on GPU clusters. You will optimize engines and GPU kernels to ensure peak model serving efficiency.

The role requires deep expertise in CUDA/Triton, inference internals, and experience with at least one serving engine. Collaboration with global teams across Rakuten will be essential.

Qualifications

• 3+ years of hands-on experience optimizing deep learning inference on NVIDIA GPUs, preferably for large language models.
• Strong knowledge of LLM inference internals: attention mechanisms, KV cache, batching strategies, quantization, and parallelism.
• Working experience with at least one mainstream inference serving engine (vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or equivalent).
• Proficiency in CUDA and/or Triton kernel programming, with a solid understanding of modern GPU architecture.
• Strong software engineering fundamentals in Python and C++.
• Bachelor's or higher degree in Computer Science, Engineering, or a related field.

Responsibilities

• Profile, optimize, and extend modern LLM inference engines (e.g., vLLM, SGLang, TensorRT-LLM) to improve throughput, latency, and GPU utilization.
• Design and implement inference-time optimizations such as quantization, KV-cache management, continuous batching, speculative decoding, and parallelism strategies (tensor / expert / pipeline).
• Develop and tune GPU kernels (CUDA, Triton) for critical operators — attention, GEMM, MoE — targeting the latest NVIDIA architectures.
• Build benchmarking and load-testing frameworks to measure end-to-end serving performance under realistic workloads and SLOs.
• Partner with infrastructure teams to deploy and scale inference on Kubernetes-based GPU clusters, including autoscaling, fault tolerance, and observability.
• Stay current with the fast-moving inference research landscape and bring cutting-edge techniques into production.

Skills

CUDA

Python

C++

GPU kernels

Inference optimization

Inference engines

Education

Bachelor's degree in CS/Engineering

Tools

Triton

Nsight Systems

PyTorch

TensorRT

Kubernetes

DeepSpeed

Job description

Rakuten Asia Pte. Ltd. in Singapore is seeking an LLM Inference Optimization Engineer to maximize the performance, efficiency, and scalability of large-scale inference workloads on GPU clusters. You will optimize engines and GPU kernels to ensure peak model serving efficiency.

The role requires deep expertise in CUDA/Triton, inference internals, and experience with at least one serving engine. Collaboration with global teams across Rakuten will be essential.