Senior Pre-Training Engineer (Re3)

Barcelona Supercomputing Center; Madrid; Presencial; EUR 90.000 - 120.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Restaurant tickets

Private health insurance

Relocation support

Flexible working hours

Descripción de la vacante

Barcelona Supercomputing Center (BSC-CNS) seeks a Senior Pre-training Engineer to lead multilingual foundation-model pre-training in a state-of-the-art HPC environment, including MareNostrum, Leonardo, and JUPITER.

You will design distributed training strategies, scale training across hundreds to thousands of GPUs, and prototype novel architectures while collaborating with infrastructure teams and researchers across Europe.

Formación

• PhD or Master’s degree in Computer Science, Artificial Intelligence or a related field.
• Extensive experience in pre-training large language models and multi-node distributed training.
• Proven expertise with distributed training frameworks and PyTorch ecosystem.
• Excellent English (written and spoken).

Responsabilidades

• Lead the technical design and execution of large-scale pre-training pipelines for multilingual foundation models on multi-node HPC infrastructure.
• Design and implement distributed training strategies (data, tensor, pipeline and sequence parallelism, ZeRO/FSDP) to scale training across GPUs.
• Apply scaling laws to guide compute-optimal decisions on model size, dataset size, and training budget.
• Research and prototype novel model architectures (Mixture-of-Experts, alternative attention/state-space mechanisms) for pre-training at scale.
• Optimize data pipelines, tokenisation and curriculum design for multilingual pre-training.
• Collaborate with HPC infrastructure teams to maximize cluster utilization, fault tolerance and checkpointing at scale.
• Mentor junior team members on distributed pre-training best practices.

Conocimientos

Distributed training

Megatron-LM

DeepSpeed

PyTorch

Python

HPC environments

English proficiency

Educación

PhD or Master’s degree in Computer Science, AI or related field

Herramientas

Megatron-LM

DeepSpeed

PyTorch FSDP

NeMo

Descripción del empleo

Starting date: 01/09/2026

Job Reference

361_26_AII_LM_RE3

Position

Senior Pre-training Engineer (RE3)

Closing Date

Monday, 31 August, ****

Reference 361_26_AII_LM_RE3

Job title Senior Pre-training Engineer (RE3)

About BSC The Barcelona Supercomputing Center - Centro Nacional de Supercomputación (BSC-CNS) is the leading supercomputing centre in Spain.

It houses MareNostrum, one of the most powerful supercomputers in Europe, was a founding and hosting member of the former European HPC infrastructure PRACE (Partnership for Advanced Computing in Europe), and is now hosted by the European Joint Undertaking for EuroHPC JU.

The mission of BSC is to research, develop and manage information technologies in order to facilitate scientific progress.

BSC combines HPC service provision and R&D into both computer and computational science (life, earth and engineering sciences) under one roof, and currently has over **** staff from 60 countries.

Look At The BSC Experience BSC?CNS YouTube Channel

Let’s stay connected with BSC Folks!

We are particularly interested for this role in the strengths and lived experiences of women and underrepresented groups to help us avoid perpetuating biases and oversights in science and IT research.

In instances of equal merit, the incorporation of the under?represented sex will be favoured.

We promote Equity, Diversity and Inclusion, fostering an environment where each and every one of us is appreciated for who we are, regardless of our differences.

We value diversity of experiences and skills, and you could bring unique perspectives to our team.

Context And Mission

The Language Modeling Team at the newly created AI Institute hosted at BSC has consolidated experience in several NLP areas, such as massive language model building, biomedical text mining, and unsupervised learning for under?resourced languages and domains.

It has been entrusted by the Spanish and the Catalan governments with the mission to develop fundamental open?source resources and technologies for Spanish and Catalan.

In connection with this, the LM team is currently in charge of two flagship projects at the national and regional level: the ALIA project, funded by the Spanish Secretariat of Digitalisation and Artificial Intelligence, and the AINA project, aimed at developing AI resources for Catalan, funded by the Catalan Digitalisation Department.

In addition, the team is a key contributor to Open Euro LLM, the European initiative to build open, multilingual foundation models trained on EuroHPC infrastructure, in collaboration with leading European research centres.

The successful candidate will have direct access to Tier?0 European supercomputers such as MareNostrum 5, Leonardo, and JUPITER.

In this context, the newly created AI Institute is looking for talent to help design and improve its pre?training operations and infrastructure.

The Language Modeling Team is looking for candidates with a background in computational linguistics with experience in Language Technologies, specifically in Deep Learning and large language model building, and possibly in other areas of Natural Language and Speech Processing.

The successful candidate will work in a highly sophisticated HPC environment, have access to state?of?the?art systems and computational infrastructures, and establish collaborations with experts in different areas at the local and international levels.

The researcher will implement innovative techniques for language modelling and evaluation in the HPC environment.

Key Duties

Lead the technical design and execution of large-scale pre?training pipelines for multilingual foundation models on multi?node HPC infrastructure (MareNostrum, Leonardo, JUPITER and other EuroHPC systems).

Design and implement distributed training strategies (data, tensor, pipeline and sequence parallelism, ZeRO/FSDP) to scale training efficiently across hundreds to thousands of GPUs.

Apply scaling laws to guide compute?optimal decisions on model size, dataset size, and training budget.

Research and prototype novel model architectures (e.g. Mixture?of?Experts, alternative attention/state?space mechanisms) for pre?training at scale.

Optimize data pipelines, tokenisation and curriculum design for multilingual, multi?node pre?training.

Collaborate closely with HPC infrastructure teams to maximise cluster utilisation, fault tolerance and checkpointing at scale.

Mentor junior team members on best practices for distributed pre?training and performance bottleneck identification.

Requirements

Education

PhD or Master’s degree in Computer Science, Artificial Intelligence or a related field.

Essential Knowledge And Professional Experience

Extensive experience in pre?training large language models, including multi?node, multi?GPU distributed training.

Proven expertise with distributed training frameworks such as Megatron?LM, DeepSpeed, PyTorch FSDP or NeMo.

Expert knowledge of Python and Deep Learning libraries including PyTorch and DeepSpeed.

Solid understanding of scaling laws and compute?optimal training strategies.

Experience with large?scale distributed training and high?performance computing (HPC) environments.

Excellent level of English (written and spoken).

Additional Knowledge And Professional Experience

Hands?on experience with European Tier?0 HPC clusters such as MareNostrum, Leonardo, or JUPITER (or equivalent large?scale supercomputing environments) will be considered a significant asset.

Fluency in Spanish is highly valued.

Knowledge of one or more of Spain's co?official languages, including Catalan, Basque and Galician is highly desirable.

Competences

Ability to optimise complex technical workflows to reduce resource consumption and iteration time.

Technical leadership skills to establish best practices in training within the group.

Strong problem?solving skills focused on performance bottlenecks and scalability.

Collaborative mindset and ability to work closely with the agents and infrastructure teams.

Conditions

The position will be located at BSC within the Directors Department.

We offer a full?time contract (35h/week), a good working environment, a highly stimulating environment with state?of?the?art infrastructure, flexible working hours, extensive training plan, restaurant tickets, private health insurance, support to the relocation procedures.

Duration: Open?ended contract due to technical and scientific activities linked to the project and budget duration.

Holidays: 22 days of holidays + 6 personal days + 24th and 31st of December per our collective agreement.

Salary: we offer a competitive salary commensurate with the qualifications and experience of the candidate and according to the cost of living in Barcelona.

Starting date: 01/09/2026.

The Recruitment Process Consists Of Two Phases

Curriculum Analysis: Evaluation of previous experience and/or scientific history, degree, training, and other professional information relevant to the position.

• 40 points.

Interview phase: The highest?rated candidates at the curriculum level will be invited to the interview phase, conducted by the corresponding department and Human Resources.

• 60 points.

A minimum of 30 points out of 60 must be obtained to be eligible for the position.

The recruitment panel will be composed of at least three people, ensuring at least 25% representation of women.

In accordance with OTM?R principles, a gender?balanced recruitment panel is formed for each vacancy at the beginning of the process.

After reviewing the content of the applications, the panel will begin the interviews, with at least one technical and one administrative interview.

At a minimum, a personality questionnaire as well as a technical exercise will be conducted during the process.

The panel will make a final decision, and all individuals who participated in the interview phase will receive feedback with details on the acceptance or rejection of their profile.

At BSC, we seek continuous improvement in our recruitment processes.

For any suggestions or comments/complaints about our recruitment processes, please contact ******.

The vacancy will remain open until a suitable candidate has been hired.

Applications will be regularly reviewed and potential candidates will be contacted.

OTM?R principles for selection processes BSC?CNS is committed to the principles of the Code of Conduct for the Recruitment of Researchers of the European Commission and the Open, Transparent and Merit?based Recruitment principles (OTM?R).

This is applied for any potential candidate in all our processes, for example by creating gender?balanced recruitment panels and recognising career breaks etc.

BSC?CNS is an equal opportunity employer committed to diversity and inclusion.

We are pleased to consider all qualified applicants for employment without regard to race, colour, religion, sex, sexual orientation, gender identity, national origin, age, disability or any other basis protected by applicable state or local law.

#J-*****-Ljbffr