Internship | Data-driven corpus mixtures for Data-Efficient LLM pretraining

TNO; Den Haag; Hybrid; EUR 6,200 - 7,500; Full time

Benefits offered by this job

Laptop provided

Internship allowance €615/month

Travel expense contribution

Jong TNO membership

Job summary

TNO invites motivated Master’s students to join our internship in Den Haag, focused on data-driven corpus mixtures for efficient LLM pretraining. You will work with the GPT-NL team to define tasks, domains and compute budgets, and to build and evaluate language models under controlled settings.

You will develop methods, such as a genetic algorithm, to predict model performance from corpus composition and validate findings in follow-up experiments.

Qualifications

• Master’s student in Artificial Intelligence, Computer Science, or Data Science with Python and ML experience.
• Experience with language-model training is helpful but not required if you are eager to learn.
• You work carefully, independently, and discuss results with others.

Responsibilities

• Define target tasks, domains, languages, and compute budget with supervisors.
• Create controlled corpus mixtures from Propella annotations and train/evaluate small language models.
• Develop a method (e.g., genetic algorithm) to predict performance from corpus composition.
• Validate predictions in new experiments and explore transfer to larger models.
• Document work and contribute to master’s thesis and corpus-building software.

Skills

Python

Machine Learning

Experimental design

Education

Master's student in AI/CS/DS

Job description

# Internship | Data-driven corpus mixtures for Data-Efficient LLM pretrainingMake your mark on our time. Become an intern at TNO!Apply now**About this position**A language model’s performance is strongly influenced by the composition of its training data. In practice, however, corpus mixtures are often designed through intuition, for example by selecting fixed proportions of code, mathematical text, or domain-specific sources.Propella provides interpretable document-level annotations for characteristics such as topic, domain, register, quality, and language. These annotations make it possible to study systematically which proportions of different data types support strong performance on specific tasks, instead of choosing those proportions manually.During this graduation internship, you will work with TNO and SURF’s GPT-NL team on an active research problem involving language models, especially relevant for low-resource languages. The scientific insights are the main outcome; a reusable framework for building corpus mixtures will support the research.**What will be your role?**Together with the supervisors, you will define the target tasks, domains, languages, and compute budget. You will then create controlled corpus mixtures from Propella annotations and train and evaluate relatively small language models under comparable conditions.Using the results, you will develop a method (e.g. a genetic algorithm or another black-box optimization algorithm) that predicts performance from corpus composition and selects promising mixtures. You will validate these predictions in new experiments and, where time and compute allow, investigate whether the conclusions transfer to larger models or different tasks. You will work reproducibly, analyze which categories provide the strongest signal, and translate your findings into a master’s thesis and supporting corpus-building software. The exact research design will be refined together with you.**What we expect from you**We are looking for a master’s student in Artificial Intelligence, Computer Science, Data Science, or a related field with experience in Python and machine learning. Experience training language models is useful, but not required if you are motivated to learn.You work carefully and independently, think critically about experimental design, and enjoy discussing results and intermediate findings with others. Curiosity, initiative, and attention to reproducibility matter more than already knowing every technique.The intended start is between November 2026 for an eight-month project and January 2027 for a six-month project. You will receive regular guidance from TNO and the academic supervisor, with access to TNO and SURF expertise in corpus annotation, data processing, and language-model training.**What you'll get in return**An internship at TNO means working in an environment where substance and impact are central. You will become part of a knowledge organisation where research and practice come together, and where experts collaborate on solutions to current societal and technological challenges.Your internship is a period in which you can discover what suits you, where your strengths lie and what you would like to learn next. You are part of a professional working environment, gain insight into how things work in practice, and have the opportunity to build experience that goes beyond this internship alone. For many students, an internship is therefore also a first step in discovering whether TNO could be a potential next step after graduation.In addition, we offer you:* A professional and innovative internship environment in which you actively contribute to societal and technological challenges, working alongside leading experts in your field.* Personal and dedicated supervision, with focus on your learning objectives, development and study assignment.* Room to develop: you gain relevant work experience, develop both your subject‐specific and professional skills, and build a valuable network.* Use of a laptop and the facilities you need to perform your work effectively.* A monthly internship allowance of € 615 for a full-time internship, for MSc, BSc and vocational education (MBO) students.* Up to eight hours of leave per internship month for a full-time internship, allowing you to balance your internship with your studies and personal life.* A contribution towards travel expenses if you are not entitled to a student travel card.* A free membership to Jong TNO: the network for young colleagues, where you can meet other TNO colleagues and participate in sports activities, professional and personal development activities, and social events such as the annual ski trip.**TNO as an employer**Our people are at the heart of TNO. Their curiosity, expertise and entrepreneurial mindset make it possible to deliver high-impact research and innovations that contribute to society’s sustainable wellbeing and prosperity. That is why we invest in an inspiring and inclusive working environment where colleagues can excel, have autonomy and continue to grow.Your talent and ambition have every opportunity to flourish at TNO. You work with experts (both within and beyond TNO), have access to advanced technology and the freedom to explore, experiment and innovate. Our strength lies in independence, reliability and collaboration. We find each other in wonder and ingenuity. We are driven to push boundaries. By working with businesses and government, and by connecting different perspectives, we strengthen our innovative capability and create responsible, meaningful results.At TNO, we believe this is our time to help society, government and business move forward faster. Together with driven colleagues, you turn knowledge into concrete innovations or ventures that truly make a difference by combining the power of science and entrepreneurship. And in doing so, you make your mark on our time.**The selection process**You can apply for this internship position until the 21st of October, 2026. After submitting your application, you will receive a confirmation by email.TNO will arrange an appropriate internship agreement. If you have any questions about the content of the internship or the application procedure, you can contact the person listed below.If you start an internship with us, we will also ask you to provide a Certificate of Conduct (VOG).Important to know before applying:* Before the start of the internship, an internship agreement must be signed. For students enrolled at a Dutch university or university of applied sciences, TNO uses the UNL-template, supplemented with several TNO‐specific agreements. For students from foreign educational institutions and MBO (secondary vocational education) programmes, the TNO internship agreement applies. TNO does not sign any other internship agreements.* Before the start of the internship, the educational institution will need to confirm in writing that: 1. you are enrolled at the educational institution for the duration of the internship, and; 2. the internship forms part of your study programme.The confirmation of educational institution takes place by signing the UNL template or forms prepared by TNO.* Interns at TNO must have a registered residential address in the Netherlands at the start of the internship. Carrying out internship activities from abroad is not possible.* Are you an international student? Please note that you must have a BSN (Dutch citizen service number) before the start of your internship. Applying for a BSN can take several weeks, so make sure to start this process well in advance.## Has this job opening sparked your interest?Then we’d like to hear from you! Please contact us for more information about the job or the selection process. To apply, please upload your CV and covering letter using the ‘apply now’ button.