Senior Inference Runtime Engineer

Bitdeer Group; Singapore; On-site; SGD 150,000 - 210,000; Full time

Job summary

Bitdeer is seeking a Senior Inference Runtime Engineer to own the performance-critical serving layer of the MaaS platform. This role focuses on making self-hosted LLMs faster, cheaper, and more stable by optimizing the runtime stack behind OpenAI- and Anthropic-compatible APIs.

You will optimize runtimes, profile bottlenecks, and lead model onboarding while collaborating with SRE and performance teams to deliver reliable, low-latency inference services at scale.

Qualifications

• 6+ years of systems, ML infrastructure, or high-performance backend engineering experience.
• Hands-on experience with LLM serving runtimes such as vLLM, Dynamo, SGLang, TensorRT-LLM, TGI, or Triton.
• Strong understanding of GPU memory, CUDA/NCCL basics, KV cache, batching, streaming, and distributed inference tradeoffs.
• Proficient in Go or Python and comfortable reading runtime source code, profiling traces, and production metrics.
• Experience operating production inference services with strict latency, availability, and cost targets.
• Able to translate low-level performance work into customer-visible reliability, latency, and margin improvements.

Responsibilities

• Optimize prefill/decode scheduling, continuous batching, KV cache behavior, speculative decoding, long-context serving, and streaming smoothness.
• Tune and operate vLLM/Dynamo/SGLang/TensorRT-LLM-style runtimes for model-specific latency, throughput, GPU utilization, and cost efficiency.
• Profile bottlenecks across GPU memory, HBM bandwidth, NCCL/network, tokenizer, frontend/proxy, and model worker paths.
• Lead high-value model onboarding, including runtime selection, tensor/pipeline parallelism, quantization, context length, and rollback strategy.
• Define runtime playbooks and safe defaults for reasoning, tool calling, multimodal, prompt cache, and provider-specific parameters.
• Partner with SRE and performance/evaluation engineers to turn benchmark findings into production runtime improvements.

Skills

Systems ML infra

LLM serving runtimes

GPU memory & CUDA basics

Go or Python

Production inference services

Tools

vLLM

Dynamo

SGLang

TensorRT-LLM

TGI

Triton

Job description

About Bitdeer

Bitdeer is a world-leading technology company for Bitcoin mining and AI cloud. Bitdeer is committed to providing comprehensive Bitcoin mining solutions for its customers. Apart from designing industry-leading ASIC chips and manufacturing mining rigs, the Group handles complex processes involved in computing across the value chain. This includes equipment procurement, transport logistics, datacenter design and construction, equipment management, and network and facility operations. Bitdeer also offers advanced cloud capabilities to customers with a high demand for artificial intelligence. Headquartered in Singapore, Bitdeer operates globally with a diversified 3 GW energy portfolio, and deploys Bitcoin mining and HPC datacenters in the United States, Bhutan, Norway, Canada, Malaysia, and Ethiopia.

About the team

We are seeking a Senior Inference Runtime Engineer to own the performance-critical serving layer of the MaaS platform. This role focuses on making self-hosted LLMs faster, cheaper, and more stable by optimizing the runtime stack behind OpenAI- and Anthropic-compatible APIs.

What you will be responsible for

• Optimize prefill/decode scheduling, continuous batching, KV cache behavior, speculative decoding, long-context serving, and streaming smoothness.
• Tune and operate vLLM/Dynamo/SGLang/TensorRT-LLM-style runtimes for model-specific latency, throughput, GPU utilization, and cost efficiency.
• Profile bottlenecks across GPU memory, HBM bandwidth, NCCL/network, tokenizer, frontend/proxy, and model worker paths.
• Lead high-value model onboarding, including runtime selection, tensor/pipeline parallelism, quantization, context length, and rollback strategy.
• Define runtime playbooks and safe defaults for reasoning, tool calling, multimodal, prompt cache, and provider-specific parameters.
• Partner with SRE and performance/evaluation engineers to turn benchmark findings into production runtime improvements.

How you will stand out

• 6+ years of systems, ML infrastructure, or high-performance backend engineering experience.
• Hands-on experience with LLM serving runtimes such as vLLM, Dynamo, SGLang, TensorRT-LLM, TGI, or Triton.
• Strong understanding of GPU memory, CUDA/NCCL basics, KV cache, batching, streaming, and distributed inference tradeoffs.
• Proficient in Go or Python and comfortable reading runtime source code, profiling traces, and production metrics.
• Experience operating production inference services with strict latency, availability, and cost targets.
• Able to translate low-level performance work into customer-visible reliability, latency, and margin improvements.

What you will experience working with us

• A culture that values authenticity and diversity of thoughts and backgrounds;
• An inclusive and respectable environment with open workspaces and exciting start-up spirit;
• Fast-growing company with the chance to network with industrial pioneers and enthusiasts;
• Ability to contribute directly and make an impact on the future of the digital asset industry;
• Involvement in new projects, developing processes/systems;
• Personal accountability, autonomy, fast growth, and learning opportunities;
• Attractive welfare benefits and developmental opportunities such as training and mentoring.

Bitdeer is committed to providing equal employment opportunities in accordance with country, state, and local laws. Bitdeer does not discriminate against employees or applicants based on conditions such as race, colour, gender identity and/or expression, sexual orientation, marital and/or parental status, religion, political opinion, nationality, ethnic background or social origin, social status, disability, age, indigenous status, and union.

#LI-ST1