AI Benchmark Architect for Agent Evaluation

CLERA; Singapore; On-site; SGD 128,000 - 218,000; Full time

Benefits offered by this job

Visa sponsorship available

Job summary

Clera Labs, Inc. is hiring for a role focused on designing and implementing rigorous benchmarks for frontier AI agents. You will work with experts to translate workflows into tasks, build scalable evaluation infrastructure, and develop metrics that reveal model performance and failure modes.

The ideal candidate has 2–4 years in research or ML engineering, strong Python, Docker, and Linux skills, and experience delivering AI benchmarks. On-site in Singapore with visa sponsorship available.

Qualifications

• 2–4 years of experience in research or ML engineering roles, with ≥2 years building AI benchmarks.
• Strong Python, Docker, and Linux skills for research or production infra.
• Hands-on experience designing, implementing, and operating evaluations for AI agents or LLMs.
• Experience building infrastructure to run models, and developing metrics and validation studies.
• Ability to collaborate with subject-matter experts and translate workflows into evaluation tasks.
• Strong writing, attention to detail, first-principles reasoning, and startup adaptability.
• Published benchmarking or evaluation research is valued; RL pipelines and data generation are relevant.

Responsibilities

• Design, implement, and maintain benchmarks for evaluating AI agents on domain-specific tasks.
• Translate real workflows into realistic tasks and evaluation criteria with experts.
• Build infrastructure for running models and agents against benchmark tasks at scale.
• Develop metrics and analyses for benchmark difficulty, reliability, and failure modes.
• Validate benchmark results against real-world evaluations and user needs.
• Write clear technical documentation and reports communicating methods and results.

Skills

AI benchmarking experience

Research engineering

Technical writing

Independent startup environment

Tools

Python

Docker

Linux

Job description

Clera Labs, Inc. is hiring for a role focused on designing and implementing rigorous benchmarks for frontier AI agents. You will work with experts to translate workflows into tasks, build scalable evaluation infrastructure, and develop metrics that reveal model performance and failure modes.

The ideal candidate has 2–4 years in research or ML engineering, strong Python, Docker, and Linux skills, and experience delivering AI benchmarks. On-site in Singapore with visa sponsorship available.