Frontier LLM Benchmark Research Engineer

nineDots.io; Dublin; On-site; EUR 90,000 - 120,000; Full time

Job summary

nineDots.io in Dublin is seeking researchers who deeply understand frontier LLMs to design and run benchmarks, evaluate model capabilities, and expose failure modes. You will focus on adversarial testing, synthetic data, and post-training techniques across several leading models, contributing to robust safety and reliability research.

The role is five days onsite in Dublin city centre within a small, ambitious team that moves quickly and values hands-on, results-driven research over theory alone.

Qualifications

• Experience building or contributing to LLM benchmarks
• Experience in LLM benchmarking or model capability evaluation
• Experience in adversarial testing or red team activities
• Experience in AI safety/model safety research
• Experience with post-training or reinforcement learning for LLMs
• Experience generating synthetic data for model training/evaluation
• Ability to design evaluation frameworks, metrics or model judges
• Research into LLM behaviour and failure modes
• Hands-on work across multiple models (e.g., Claude, GPT, Gemini, Llama, DeepSeek)
• Actual work delivering benchmarks and evaluation systems rather than terminology only

Responsibilities

• Design and build benchmarks for frontier LLMs
• Compare model performance and behavior across models
• Create tasks and datasets to reveal strengths and weaknesses
• Adversarially test models and identify failure points
• Generate synthetic data for training and evaluation pipelines
• Investigate model reasoning, reliability and unexpected behaviour
• Work on post-training and reinforcement learning for LLMs
• Run experiments across multiple frontier models

Skills

LLM benchmarking

Model evaluation

Adversarial testing

AI safety research

Post-training / RL for LLMs

Synthetic data generation

Evaluation frameworks

Model behavior research

Cross-model experiments

Hands-on frontier model work

Education

PhD in a relevant area

Job description

nineDots.io in Dublin is seeking researchers who deeply understand frontier LLMs to design and run benchmarks, evaluate model capabilities, and expose failure modes. You will focus on adversarial testing, synthetic data, and post-training techniques across several leading models, contributing to robust safety and reliability research.

The role is five days onsite in Dublin city centre within a small, ambitious team that moves quickly and values hands-on, results-driven research over theory alone.