Senior LLM Inference Performance Engineer

Bitdeer Group; Singapore; On-site; SGD 150,000 - 210,000; Full time

Job summary

Bitdeer is seeking a Senior Inference Runtime Engineer to own the performance-critical serving layer of the MaaS platform. This role focuses on making self-hosted LLMs faster, cheaper, and more stable by optimizing the runtime stack behind OpenAI- and Anthropic-compatible APIs.

You will optimize runtimes, profile bottlenecks, and lead model onboarding while collaborating with SRE and performance teams to deliver reliable, low-latency inference services at scale.

Qualifications

• 6+ years of systems, ML infrastructure, or high-performance backend engineering experience.
• Hands-on experience with LLM serving runtimes such as vLLM, Dynamo, SGLang, TensorRT-LLM, TGI, or Triton.
• Strong understanding of GPU memory, CUDA/NCCL basics, KV cache, batching, streaming, and distributed inference tradeoffs.
• Proficient in Go or Python and comfortable reading runtime source code, profiling traces, and production metrics.
• Experience operating production inference services with strict latency, availability, and cost targets.
• Able to translate low-level performance work into customer-visible reliability, latency, and margin improvements.

Responsibilities

• Optimize prefill/decode scheduling, continuous batching, KV cache behavior, speculative decoding, long-context serving, and streaming smoothness.
• Tune and operate vLLM/Dynamo/SGLang/TensorRT-LLM-style runtimes for model-specific latency, throughput, GPU utilization, and cost efficiency.
• Profile bottlenecks across GPU memory, HBM bandwidth, NCCL/network, tokenizer, frontend/proxy, and model worker paths.
• Lead high-value model onboarding, including runtime selection, tensor/pipeline parallelism, quantization, context length, and rollback strategy.
• Define runtime playbooks and safe defaults for reasoning, tool calling, multimodal, prompt cache, and provider-specific parameters.
• Partner with SRE and performance/evaluation engineers to turn benchmark findings into production runtime improvements.

Skills

Systems ML infra

LLM serving runtimes

GPU memory & CUDA basics

Go or Python

Production inference services

Tools

vLLM

Dynamo

SGLang

TensorRT-LLM

TGI

Triton

Job description

Bitdeer is seeking a Senior Inference Runtime Engineer to own the performance-critical serving layer of the MaaS platform. This role focuses on making self-hosted LLMs faster, cheaper, and more stable by optimizing the runtime stack behind OpenAI- and Anthropic-compatible APIs.

You will optimize runtimes, profile bottlenecks, and lead model onboarding while collaborating with SRE and performance teams to deliver reliable, low-latency inference services at scale.