Senior LLM Inference Performance & Evaluation Engineer

Bitdeer Group; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Bitdeer is seeking an experienced ML infrastructure engineer to build and maintain benchmark pipelines, define model launch gates, and simulate realistic workloads at scale.

You will evaluate model and runtime options, implement regression detection in CI/CD, and collaborate with SRE to turn benchmarks into actionable SLOs and alerts. The role emphasizes performance, cost, and reliability in a fast‑growing AI and mining tech environment.

Qualifications

• 5+ years in ML infrastructure, performance engineering, model evaluation, QA automation, or backend testing for production systems.
• Experience with LLM serving metrics such as TTFT, TPOT/ITL, request latency, token throughput, concurrency, and GPU utilization.
• Strong Python skills for benchmark/evaluation automation; Go experience is a plus for integrating with platform services.
• Familiarity with OpenAI/Anthropic APIs, vLLM/Dynamo/SGLang/Triton‑style servers, and Kubernetes‑based test environments.
• Able to design statistically meaningful tests and communicate tradeoffs between model quality, latency, reliability, cost, and customer experience.
• Experience building dashboards, reports, and release gates for engineering, product, and business stakeholders.

Responsibilities

• Build benchmark pipelines for TTFT, inter‑token latency, output throughput, request latency, concurrency curves, error rate, and GPU utilization.
• Create model launch gates covering OpenAI/Anthropic compatibility, streaming behavior, tool calling, reasoning output, multimodal behavior, and long‑context cases.
• Maintain representative workloads using synthetic, replayed, and customer‑like traffic patterns for both steady‑state and burst scenarios.
• Compare model/runtime/provider options and publish clear recommendations for routing, fallback, pricing, and capacity decisions.
• Automate regression detection in CI/CD and staging so model, runtime, or configuration changes do not silently degrade quality, latency, or cost.
• Work with runtime engineers to identify bottlenecks and verify improvements; work with SRE to convert benchmark results into SLOs and alert thresholds.

Skills

Python

OpenAI API

Anthropic API

Benchmarking

Model evaluation

QA automation

Performance engineering

Go

Kubernetes

vLLM

Dynamo

SGLang

Triton

Tools

vLLM

Dynamo

SGLang

Triton

Job description

About Bitdeer:

Bitdeer is a world‑leading technology company for Bitcoin mining and AI cloud. Bitdeer is committed to providing comprehensive Bitcoin mining solutions for its customers. Apart from designing industry‑leading ASIC chips and manufacturing mining rigs, the Group handles complex processes involved in computing across the value chain. This includes equipment procurement, transport logistics, datacenter design and construction, equipment management, and network and facility operations. Bitdeer also offers advanced cloud capabilities to customers with a high demand for artificial intelligence.

Headquartered in Singapore, Bitdeer operates globally with a diversified 3 GW energy portfolio, and deploys Bitcoin mining and HPC datacenters in the United States, Bhutan, Norway, Canada, Malaysia, and Ethiopia.

What you will be responsible for:

• Build benchmark pipelines for TTFT, inter‑token latency, output throughput, request latency, concurrency curves, error rate, and GPU utilization.
• Create model launch gates covering OpenAI/Anthropic compatibility, streaming behavior, tool calling, reasoning output, multimodal behavior, and long‑context cases.
• Maintain representative workloads using synthetic, replayed, and customer‑like traffic patterns for both steady‑state and burst scenarios.
• Compare model/runtime/provider options and publish clear recommendations for routing, fallback, pricing, and capacity decisions.
• Automate regression detection in CI/CD and staging so model, runtime, or configuration changes do not silently degrade quality, latency, or cost.
• Work with runtime engineers to identify bottlenecks and verify improvements; work with SRE to convert benchmark results into SLOs and alert thresholds.

How you will stand out:

• 5+ years in ML infrastructure, performance engineering, model evaluation, QA automation, or backend testing for production systems.
• Experience with LLM serving metrics such as TTFT, TPOT/ITL, request latency, token throughput, concurrency, and GPU utilization.
• Strong Python skills for benchmark/evaluation automation; Go experience is a plus for integrating with platform services.
• Familiarity with OpenAI/Anthropic APIs, vLLM/Dynamo/SGLang/Triton‑style servers, and Kubernetes‑based test environments.
• Able to design statistically meaningful tests and communicate tradeoffs between model quality, latency, reliability, cost, and customer experience.
• Experience building dashboards, reports, and release gates for engineering, product, and business stakeholders.

What you will experience working with us:

• A culture that values authenticity and diversity of thoughts and backgrounds;
• An inclusive and respectable environment with open workspaces and exciting start‑up spirit;
• Fast‑growing company with the chance to network with industrial pioneers and enthusiasts;
• Ability to contribute directly and make an impact on the future of the digital asset industry;
• Involvement in new projects, developing processes/systems;
• Personal accountability, autonomy, fast growth, and learning opportunities;
• Attractive welfare benefits and developmental opportunities such as training and mentoring.

Bitdeer is committed to providing equal employment opportunities in accordance with country, state, and local laws. Bitdeer does not discriminate against employees or applicants based on conditions such as race, colour, gender identity and/or expression, sexual orientation, marital and/or parental status, religion, political opinion, nationality, ethnic background or social origin, social status, disability, age, indigenous status, and union.