Applied Scientist: Agent Evaluation & Adaptive Routing

Bitdeer; Austin, Northern (TX, KY); Hybrid; USD 125,000 - 170,000; Full time

Benefits offered by this job

Mentoring program

Training opportunities

Competitive benefits

Job summary

Bitdeer AI Lab in Austin is seeking a senior researcher to build evaluation pipelines for agentic inference and to extend our LLM and agent evaluation frameworks. You will design task-level and trajectory-level metrics, prototype routing strategies, and drive production-ready research artifacts.

The role requires deep expertise in Python, PyTorch, and evaluating multi-turn, tool-using agents, with a track record of robust experimental design and open-source contributions.

Qualifications

• Hands-on experience in LLM evaluation and adaptive inference.
• Strong Python programming and PyTorch experience; able to build reliable experimental pipelines.
• Experience designing evaluation pipelines for LLMs or agents, including task-level and trajectory-level metrics.
• Depth in model selection and routing, uncertainty estimation/calibration, cascading/escalation, or stage-aware inference.

Responsibilities

• Build evaluation and decision systems for agentic inference; own LLM/agent evaluation pipeline.
• Research and prototype adaptive model-routing strategies and escalation policies.
• Collaborate with MaaS and platform teams on production integration.

Skills

Python programming

LLM evaluation

Agentic systems

Uncertainty estimation

Education

Bachelor’s/Master’s/PhD in CS/ML/Statistics/EE

Tools

PyTorch

vLLM

SGLang

Job description

Bitdeer AI Lab in Austin is seeking a senior researcher to build evaluation pipelines for agentic inference and to extend our LLM and agent evaluation frameworks. You will design task-level and trajectory-level metrics, prototype routing strategies, and drive production-ready research artifacts.

The role requires deep expertise in Python, PyTorch, and evaluating multi-turn, tool-using agents, with a track record of robust experimental design and open-source contributions.