Applied Scientist, Agent Evaluation & Adaptive Routing

Bitdeer; Singapore; On-site; SGD 180,000 - 260,000; Full time

Benefits offered by this job

Culture of authenticity

Open workspaces

Startup spirit

Training & mentoring

Welfare benefits

Job summary

Bitdeer is seeking a research-minded professional to build evaluation systems for agentic inference and own the LLM evaluation pipeline and routing policies. You will design task-level and trajectory-level metrics, prototype adaptive routing strategies, and contribute to scalable experiments with industry-grade rigor.

Works closely with MaaS and core platform teams to ensure production-quality integration, monitoring, and cost–quality tradeoffs.

Qualifications

• Bachelor’s, Master’s, or PhD in CS, ML, Statistics, or EE with hands-on evaluation experience.
• Strong Python and PyTorch; build reliable experimental pipelines.
• Experience designing evaluation pipelines for LLMs or agents with task and trajectory metrics.
• Deep knowledge in model routing, uncertainty estimation, cascading, escalation, or stage-aware agent inference.
• Experience with online learning, contextual bandits, or adaptive inference.
• Hands-on evaluation of multi-turn or tool-using agents, focusing on cost, latency, and correctness.
• Rigorous experimental practice with proper baselines and statistical analysis.
• Ability to analyze per-request trajectory quality and production monitoring concepts.

Responsibilities

• Develop evaluation systems for agentic inference and measurable performance.
• Own LLM evaluation pipeline and routing policies; prototype research models.
• Collaborate with MaaS and platform teams for production integration and SLA reliability.
• Experiment with offline evaluation, traffic replay, and shadow testing.
• Publish findings and build reproducible experiments.

Skills

Python

PyTorch

LLM evaluation

Agentic systems

Experiment pipelines

Model routing

Uncertainty estimation

A/B testing

Tool-use evaluation

Education

PhD in CS/ML

Master's degree

Bachelor's degree

Tools

vLLM

SGLang

Evaluation tooling

Dataset construction

Job description

Bitdeer is seeking a research-minded professional to build evaluation systems for agentic inference and own the LLM evaluation pipeline and routing policies. You will design task-level and trajectory-level metrics, prototype adaptive routing strategies, and contribute to scalable experiments with industry-grade rigor.

Works closely with MaaS and core platform teams to ensure production-quality integration, monitoring, and cost–quality tradeoffs.