AI Benchmarking Research Engineer

Hud; Singapore; Remote; SGD 128,000 - 224,000; Full time

Benefits offered by this job

Medical, dental, vision benefits (US &

Job summary

HUD is seeking Research Engineers to build high-quality benchmarks for evaluating frontier agents on domain-specific tasks. You will design, implement, and maintain benchmarks, collaborating with experts to ensure realism and usefulness across various workflows.

Ideal candidates will be proficient in Python, Docker, and Linux, with a track record of publications or technical blogs on comparable benchmarks and failure modes.

Qualifications

• Proficiency in Python, Docker, and Linux environments.
• Publications or technical blogs on benchmarks, model failures, or related topics.

Responsibilities

• Own the design, implementation, and quality of HUD’s internal agent benchmarks.
• Define tasks with experts to create domain-specific benchmarks evaluating agents on realistic workflows.
• Build infrastructure to run models and agents against benchmark tasks.
• Develop metrics and analyses to understand benchmark difficulty and failure modes.
• Validate benchmark performance against real-world evals, customer needs, and lab expectations.
• Write documentation and benchmark reports for technical audiences.

Skills

Python

Docker

Linux

Publications/blogs

Job description

HUD is seeking Research Engineers to build high-quality benchmarks for evaluating frontier agents on domain-specific tasks. You will design, implement, and maintain benchmarks, collaborating with experts to ensure realism and usefulness across various workflows.

Ideal candidates will be proficient in Python, Docker, and Linux, with a track record of publications or technical blogs on comparable benchmarks and failure modes.