Senior Software Engineer

Rakuten Asia Pte Ltd; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Rakuten Asia Pte Ltd in Singapore invites an experienced LLM Inference Optimization Engineer to join our AI & Data Division. You will optimize large-scale inference workloads on GPU clusters, implement quantization and caching strategies, and collaborate with global teams to maximize throughput and minimize latency.

You will work with vLLM, TensorRT-LLM, Triton, and custom kernels on NVIDIA hardware, while profiling performance and driving SLO-aligned improvements in a fast-paced, international

Qualifications

• 3+ years of hands-on experience optimizing deep learning inference on NVIDIA GPUs.
• Strong knowledge of LLM inference internals: attention, KV cache, batching, quantization.
• Experience with at least one mainstream inference engine (vLLM, SGLang, TensorRT-LLM, Triton).
• Proficiency in CUDA and/or Triton kernel programming, with a solid understanding of modern GPU architecture.
• Strong software engineering fundamentals in Python and C++.
• Bachelor's or higher degree in Computer Science, Engineering, or related field.

Responsibilities

• Profile, optimize, and extend modern LLM inference engines (e.g., vLLM, SGLang, TensorRT-LLM) to improve throughput, latency, and GPU utilization.
• Design and implement inference-time optimizations such as quantization, KV-cache management, continuous batching, speculative decoding, and parallelism strategies.
• Develop and tune GPU kernels (CUDA, Triton) for critical operators — attention, GEMM, MoE.
• Build benchmarking and load-testing frameworks to measure end-to-end serving performance under realistic workloads and SLOs.
• Partner with infrastructure teams to deploy and scale inference on Kubernetes-based GPU clusters, including autoscaling, fault tolerance, and observability.

Skills

LLM inference

CUDA / Triton

Python

C++

GPU profiling

Education

Bachelor's degree in CS/Engineering

Tools

vLLM

SGLang

TensorRT-LLM

Triton Inference Server

Nsight Systems

Job description

Situated in the heart of Singapore's Central Business District, Rakuten Asia Pte. Ltd. is Rakuten's Asia Regional headquarters. Established in August 2012 as part of Rakuten's global expansion strategy, Rakuten Asia comprises various businesses that provide essential value-added services to Rakuten's global ecosystem. Through advertisement product development, product strategy, and data management, among others, Rakuten Asia is strengthening Rakuten Group's core competencies to take the lead in an increasingly digitalized world. AI & Data Division (AIDD) spearheads data science & AI initiatives by leveraging data from Rakuten Group. We build a platform for large-scale field experimentations using cutting-edge technologies to provide critical insights that enable faster and better and faster contribution for our business. Our division boasts an international culture created by talented employees from around the world. Following the strategic vision “Rakuten as a data-driven membership company”, AIDD is expanding its data & AI related activities across multiple Rakuten Group companies. As an LLM Inference Optimization Engineer, you will focus on maximizing the performance, efficiency, and scalability of LLM inference workloads on Rakuten's GPU clusters. You will deeply optimize inference engines (e.g., vLLM, SGLang, TensorRT-LLM) along with their underlying GPU kernels and runtimes, ensuring Rakuten's AI models are served at peak efficiency. This role requires strong expertise in GPU-accelerated ML systems and inference optimization, with a focus on reducing latency, improving throughput, and maximizing GPU utilization for large-scale LLM serving.

Key Responsibilities

• Profile, optimize, and extend modern LLM inference engines (e.g., vLLM, SGLang, TensorRT-LLM) to improve throughput, latency, and GPU utilization.
• Design and implement inference-time optimizations such as quantization, KV-cache management, continuous batching, speculative decoding, and parallelism strategies (tensor / expert / pipeline).
• Develop and tune GPU kernels (CUDA, Triton) for critical operators — attention, GEMM, MoE — targeting the latest NVIDIA architectures.
• Build benchmarking and load-testing frameworks to measure end-to-end serving performance under realistic workloads and SLOs.
• Partner with infrastructure teams to deploy and scale inference on Kubernetes-based GPU clusters, including autoscaling, fault tolerance, and observability.
• Stay current with the fast-moving inference research landscape and bring cutting-edge techniques into production.

Mandatory Qualifications

• 3+ years of hands-on experience optimizing deep learning inference on NVIDIA GPUs, preferably for large language models.
• Strong knowledge of LLM inference internals: attention mechanisms, KV cache, batching strategies, quantization, and parallelism.
• Working experience with at least one mainstream inference serving engine (vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or equivalent).
• Proficiency in CUDA and/or Triton kernel programming, with a solid understanding of modern GPU architecture.
• Strong software engineering fundamentals in Python and C++.
• Bachelor's or higher degree in Computer Science, Engineering, or a related field.

Nice-to-Have Skills

• Experience contributing to open-source inference or ML systems projects.
• Familiarity with advanced inference techniques such as MoE serving, disaggregated prefill/decode, prefix caching, or long-context optimization.
• Experience with low-precision inference (FP8, INT4, FP4) and model compression workflows.
• Hands-on experience with GPU performance profiling tools (Nsight Systems / Compute, PyTorch Profiler).
• Experience deploying and operating large-scale inference services on Kubernetes.
• Understanding of SLO-driven capacity planning and cost optimization for online AI services.
• Experience developing or contributing to deep learning training frameworks (e.g., PyTorch, DeepSpeed, Megatron-LM, FSDP) is a plus, especially for candidates who can bridge training-time decisions with inference-time deployment.

Why Join Us?

Work on cutting‑edge LLM inference optimization at scale. Directly impact Rakuten's AI infrastructure by improving efficiency and reducing costs. Collaborate with global AI/ML teams on high-impact challenges. Opportunity to research and implement state‑of‑the‑art GPU optimizations.

Rakuten is an equal opportunities employer and welcomes applications regardless of sex, marital status, ethnic origin, sexual orientation, religious belief, or age.

Rakuten Asia is the regional headquarters of Rakuten Group Inc, a global leader in internet services and innovation. Located in the heart of Singapore, Rakuten Asia drives the growth and development of Rakuten’s businesses across the APAC region. Recognized and certified as a "Great Place to Work" since 2022, Rakuten Asia is committed to empowering individuals, communities and businesses through cutting-edge technology, data-driven solutions, and a culture of collaboration. From e-commerce, fintech, sports, and even entertainment, our work is diverse, exciting, and impactful. We believe in empowering our people to grow, innovate, and make a difference. If you’re looking for a workplace where your talent is valued, where your contributions matter, and where you can play a part in shaping the future of technology and society, Rakuten Asia is the place for you. Join us and take the next step in your career with Rakuten Asia!