Lead AI Evaluation & Research Architect

FUNDA; Singapore; On-site; SGD 180,000 - 240,000; Full time

Job summary

FUNDA is building AI-native research systems for professional users. We are looking for a Lead AI Research Engineer to establish evaluation foundations for evaluating our LLM and agent systems.

The role spans applied AI research, experimental science, and production architecture, with emphasis on empirical evidence guiding decisions across models, prompts, tools, data, and agent design. The candidate will define evaluation strategies, design robust benchmarks, and translate findings into

Qualifications

• Advanced training or equivalent demonstrated expertise in Computer Science, Artificial Intelligence, Machine Learning, Statistics, or another highly quantitative discipline.
• Strong track record of conducting applied research and building sophisticated LLM, agent, or machine-learning evaluation systems.
• Deep understanding of experimental design, benchmark construction, statistical reasoning, error analysis, measurement validity, and reproducibility.
• Strong software and systems engineering ability, with experience translating research ideas into reliable production infrastructure.
• Demonstrated ability to evaluate probabilistic systems where correctness cannot be captured by a single metric or static test set.

Responsibilities

• Define FUNDA’s evaluation strategy, technical architecture, and research methodology for production LLM and agent systems.
• Design evaluation frameworks that faithfully represent real agent behavior, including multi-step reasoning, tool use, information retrieval, long-running execution, and generated artifacts.
• Develop benchmarks that measure not only final-answer quality, but also reasoning trajectories, evidence use, task completion, robustness, and operational reliability.
• Create advanced evaluation methods combining deterministic verification, task-specific grading, process supervision, model-based evaluation, statistical analysis, and targeted expert review.
• Lead controlled experiments and comparative studies across models, prompts, tools, data strategies, and agent architectures.
• Establish rigorous standards for dataset construction, baseline selection, error taxonomy, experiment reproducibility, statistical validity, and regression detection.
• Diagnose complex system failures and distinguish whether they originate from model capability, context construction, tool behavior, data quality, orchestration logic, or infrastructure.
• Develop methods for identifying benchmark contamination, evaluator bias, unstable metrics, and differences between offline evaluation and production performance.
• Translate research findings into architectural recommendations, release criteria, and prioritised engineering investments.
• Build reusable evaluation infrastructure capable of supporting reproducible experiments across different execution environments and agent configurations.
• Set technical standards and mentor engineers in evaluation-driven AI development, experimental reasoning, and evidence-based decision-making.
• Track relevant advances in LLM evaluation, agent benchmarking, AI reliability, and model supervision, and determine which methods are suitable for production adoption.

Skills

Applied AI research

LLM evaluation

Experimental design

Production infrastructure

Statistical reasoning

Python

PyTorch

Education

Advanced degree in CS/AI/ML or related

Tools

Python

PyTorch

Job description

FUNDA is building AI-native research systems for professional users. We are looking for a Lead AI Research Engineer to establish evaluation foundations for evaluating our LLM and agent systems.

The role spans applied AI research, experimental science, and production architecture, with emphasis on empirical evidence guiding decisions across models, prompts, tools, data, and agent design. The candidate will define evaluation strategies, design robust benchmarks, and translate findings into