Software Engineer, LLM Evaluation – English

Blue Lynx Employment BV; Netherlands; On-site; EUR 60,000 - 80,000; Full time

Benefits offered by this job

25 holidays per annum

Pension plan

Opportunity to work on cutting-edge AI initiatives

Job summary

Blue Lynx Employment BV is seeking a Software Engineer for LLM evaluation to join a remote team in the Netherlands. The role involves analyzing large language models' performance, conducting data-driven analyses, and contributing to innovative AI initiatives.

The ideal candidate will have a strong background in AI, Machine Learning, and programming in Python, with a focus on model evaluation. The position offers opportunities to work with experienced researchers in a collaborative environment.

Qualifications

• 3–5 years of experience working with large language models.
• Strong hands-on programming experience in Python.
• Demonstrated scientific curiosity and problem-solving skills.

Responsibilities

• Analyse and evaluate large language models' performance.
• Execute benchmark evaluations and generate performance metrics.
• Develop and maintain ML/DL systems and tooling.

Skills

Fluent in English

Programming in Python

Experience with large language models

ML/DL systems maintenance

Research in AI

Data analysis

Education

Bachelor's degree in Computer Science or related

Master's or PhD in relevant field

Tools

PyTorch

Job description

Software Engineer, LLM Evaluation – English

Job Specs

Category: Software Engineering/Development, Artificial Intelligence

Region: Netherlands: Remote

Depends on experience

Hiring type: Contracting

Duration: 1 year with view to extension

ASAP

Job ref. 26-0418

Description

Our client’s mission is to empower people, build community, and bring the world closer together. Through their apps and services, they are building a different kind of company that connects people worldwide and provides meaningful ways to share what matters most.

We are looking for a Software Engineer, LLM Evaluation, to join their team in the Netherlands remotely.

Job Profile: The successful candidate will work closely with researchers and engineers to evaluate model performance, conduct data‑driven analyses, and contribute to research initiatives related to LLM pretraining and evaluation.

Responsibilities

• Analyse and evaluate large language models and their performance across various tasks and benchmarks
• Execute benchmark evaluations and generate performance metrics and insights
• Conduct quantitative and qualitative data analysis to support research objectives
• Contribute to the design, implementation, and validation of new evaluation methodologies
• Support research initiatives related to LLM pretraining and model evaluation
• Develop and maintain machine learning and deep learning systems and tooling
• Build research workflows and experimental frameworks using Python and PyTorch
• Enable rapid experimentation and support the execution of research initiatives
• Collaborate with researchers and engineers within a multidisciplinary team environment

Qualifications

• Must be fluent in English, both written and spoken
• Bachelor’s degree in Computer Science, Artificial Intelligence, Machine Learning, or a related technical discipline. Master’s degree in Computer Science, Machine Learning, AI, or a related discipline is desirable; a PhD in a relevant field would be considered a strong advantage
• 3–5 years of experience working with large language models, including pretraining and evaluation
• Strong hands‑on programming experience in Python
• Proven experience building and maintaining ML/DL systems and research infrastructure
• Experience developing machine learning and deep learning solutions using PyTorch
• Publications in machine learning, natural language processing, artificial intelligence, or related fields are a plus
• Experience working with transformer architectures, large language models, and/or multimodal models is an advantage
• Ability to write clean, efficient, and production-quality code
• Strong interest in model evaluation, experimentation, and data analysis
• Demonstrated scientific curiosity and problem‑solving skills

Benefits

• 25 holidays per annum
• Pension plan
• Opportunity to work alongside experienced researchers and engineers on cutting‑edge AI initiatives
• Exposure to state‑of‑the‑art methodologies and technologies within the LLM ecosystem
• Collaborative and research‑driven environment in a technologically advanced office

To learn more about the role, please visit our website.