Master Thesis: Cross-Lingual Safety and System Identity Alignment via DPO

AI Sweden; Stockholms kommun; Hybrid; SEK 246,000 - 368,000; Full time

Benefits offered by this job

Hybrid/Remote options

Job summary

AI Sweden invites MSc students to join as thesis researchers focused on open-weight European AI models, identity consistency, and safety alignment across languages. You’ll design experiments, run evaluations, and contribute to a growing body of public research data.

Location is Hybrid (Gothenburg / Stockholm) or Remote. You will collaborate with leading AI scientists at AI Sweden, aiming for high-impact publications and practical demonstrations that advance Swedish industry and the public

Qualifications

• Pursuing MSc in Computer Science, Data Science, ML, or related field.
• Strong Python programming and practical experience with deep learning frameworks.
• Familiarity with LLM post-training alignment (SFT, DPO, RLHF) or context-extension.

Responsibilities

• Design experiments to assess identity consistency and safety across languages.
• Analyze and curate data, evaluate model behaviors, and compare training approaches.
• Implement and evaluate alignment strategies using open-weight models and multilingual datasets.

Skills

Python

PyTorch

Hugging Face

Distributed GPU

Education

Ongoing Master’s studies in Computer Science

Tools

Linux HPC

Job description

Introduction

Open-weight multilingual models should maintain a consistent system identity and appropriate safety behaviour across languages. However, safety alignment can vary across languages and may introduce over-refusal, particularly in lower-resource settings.

Project Background and Problem Statement

We have developed the openeurollm-model-identity dataset and a multilingual safety refusal suite. This thesis investigates DPO-based alignment in Swedish and Icelandic, focusing on three questions:

• System identity: Does the model reliably identify itself as OpenEuroLLM and resist attempts to overwrite that identity?
• Safety alignment: Does the model appropriately refuse genuinely unsafe requests across languages?
• Over-refusal: Does safety alignment cause the model to refuse legitimate requests?

The broader question is whether these behaviours can be improved without degrading general instruction-following performance.

Outline

• Literature study: Review cross-lingual safety alignment, over-refusal, model identity, and preference optimization.
• Data analysis: Assess the quality and suitability of the preference data, including both preference correctness and the absolute quality of chosen responses.
• Implementation: Fine-tune Prelude 9B using identity and safety preference data with Hugging Face TRL or Megatron-LM. Compare identity-only, safety-only, and combined training where feasible.
• Evaluation: Compare the resulting models against the baseline on identity consistency, safety and jailbreak robustness, over-refusal on benign queries, and general instruction-following performance in Swedish and Icelandic.

Who We’re Looking For

We are seeking curious, self-driven MSc students eager to work at the frontier of open-weight European AI research (LLMs). You thrive on empirical discovery, design rigorous experiments, and let data challenge your assumptions.

• Ongoing Master’s studies in Computer Science, Data Science, Machine Learning, Engineering Physics, or a related quantitative field.
• Proficiency in Python and hands‑on experience with modern deep learning frameworks (PyTorch, Hugging Face ecosystem).
• Familiarity with LLM post‑training alignment (e.g., SFT, DPO, RLHF/RLVR) or context‑extension, alongside comfort running distributed GPU training in Linux/HPC environments.

At AI Sweden, we are committed to building diverse and inclusive teams. Some positions may be subject to export control regulations, which means that specific requirements may apply.

Why should you do your thesis with AI Sweden?

Doing your thesis at AI Sweden means working alongside leading AI scientists and change leaders. AI Sweden is Sweden’s National Center for AI, we drive research questions that have both a long shelf-life and are widely applicable to Swedish industry and the public sector. We aim for publications at the most competitive venues and celebrate a culture of research excellence.

As an organization, we’re uniquely positioned at the sweet spot of governmental influence and startup agility. Small enough to stay adaptive and have fun but backed by and in close contact with both the government, academia and private and public sector.

Practical details

Location: Hybrid (Gothenburg / Stockholm) or Remote.

Application Deadline: 2026-11-20 (rolling selection – position may be filled earlier).

Start Date: January 2027

Contact

If you have any questions or thoughts, don’t hesitate to contact:

Amaru Cuba Gyllensten, Senior Research Scientist

Danila Petrelli, Senior Data Lead/Research Scientist

References

1. [1] OpenEuroLLM Consortium, \"openeurollm-model-identity & Safety Evaluation Suite\", 2026.
2. [2] PKU Alignment Team, \"Safe-RLHF: Safe Reinforcement Learning from Human Feedback\", 2024