Applied Scientist: Agent Evaluation & Adaptive Routing

Bitdeer; Singapore; On-site; SGD 180,000 - 260,000; Full time

Job summary

Bitdeer AI Lab seeks a scientist to build and operate evaluation systems for agentic inference. You will own LLM evaluation pipelines, craft task suites, and measure behavior across fidelity, cost, latency, and token use.

Collaboration with MaaS and platform teams enables production integration and scalable research. You will push routing policies, uncertainty estimation, and stage-aware agent inference, while driving robust experimentation and transparent results.

Qualifications

• Hands-on experience in LLM evaluation, agentic systems, or adaptive inference.
• Strong Python programming with PyTorch and evaluation tooling.
• Experience designing evaluation pipelines for LLMs or agents.
• Depth in model selection, routing, uncertainty estimation, or escalation.
• Experience evaluating multi-turn or tool-using agents.
• Rigorous experimental practice with honest baselines.

Responsibilities

• Build evaluation and decision systems that measure agentic inference.
• Own and extend LLM evaluation pipeline and task suites.
• Research routing strategies and production integration with MaaS and platform teams.
• Ensure production gateway reliability, billing, access control, and SLA engineering.

Skills

LLM evaluation

Agentic systems

Python

PyTorch

Experimental pipelines

Routing policy

Uncertainty estimation

AB testing

Cost/latency analysis

Multi-model APIs

Education

Bachelor’s/Master’s/PhD in CS/ML

Tools

vLLM

SGLang

Job description

Bitdeer AI Lab seeks a scientist to build and operate evaluation systems for agentic inference. You will own LLM evaluation pipelines, craft task suites, and measure behavior across fidelity, cost, latency, and token use.

Collaboration with MaaS and platform teams enables production integration and scalable research. You will push routing policies, uncertainty estimation, and stage-aware agent inference, while driving robust experimentation and transparent results.