Frontier LLM Benchmark Research Engineer

nineDots.io; Dublin; On-site; EUR 120,000 - 180,000; Full time

Job summary

nineDots.io in Dublin is seeking an AI Research Engineer focused on frontier models and benchmarking. You will design and run rigorous evaluation protocols across multiple state-of-the-art LLMs and analyze model behavior.

The role values deep research credentials, publications or demonstrated benchmark work, and the ability to build evaluation systems and datasets. You’ll work on post-training and reinforcement learning experiments and compare model strengths across several families.

Qualifications

• Deep expertise in frontier LLM benchmarks and model evaluation.
• Publications or demonstrable benchmark work.
• Experience building evaluation systems and datasets.

Responsibilities

• Design benchmarks for frontier LLMs.
• Compare model performance across families such as Claude, GPT, Gemini, Llama.
• Develop datasets to expose strengths and weaknesses.
• Conduct adversarial testing and reliability analyses.
• Work on post-training and reinforcement learning experiments.

Skills

LLM benchmarking

Adversarial testing

AI safety research

Post-training RL

Synthetic data generation

Evaluation frameworks

Model evaluation

Multimodel analysis

Education

PhD in a relevant area

Publications at conferences (NeurIPS/ICML/ICLR)

Job description

nineDots.io in Dublin is seeking an AI Research Engineer focused on frontier models and benchmarking. You will design and run rigorous evaluation protocols across multiple state-of-the-art LLMs and analyze model behavior.

The role values deep research credentials, publications or demonstrated benchmark work, and the ability to build evaluation systems and datasets. You’ll work on post-training and reinforcement learning experiments and compare model strengths across several families.