Senior Pre-Training Engineer for Multilingual LLMs (HPC)

Barcelona Supercomputing Center; Barcelona; Presencial; EUR 90.000 - 120.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Restaurant tickets

Private health insurance

Relocation support

Descripción de la vacante

The Barcelona Supercomputing Center (BSC-CNS) invites applications for a Senior Pre-training Engineer (RE3) to lead multilingual language model pre-training and scalable HPC workflows. You will design distributed training strategies, work with Megatron-LM, DeepSpeed and PyTorch, and collaborate with Tier-0 European supercomputing resources.

The role requires a PhD or Master in CS/AI, expert Python and DL libraries, strong English, and proven HPC experience.

Formación

• PhD or Master's in Computer Science, AI or related field.
• Extensive experience with large language model pre-training and multi-node, multi-GPU training.
• Proficiency with Megatron-LM, DeepSpeed, PyTorch, and NeMo.

Responsabilidades

• Lead the design and execution of large-scale multilingual pre-training pipelines on multi-node HPC infrastructure.
• Design distributed training strategies (data, tensor, pipeline, and sequence parallelism) to scale across hundreds/thousands of GPUs.
• Research and prototype novel model architectures for pre-training at scale.
• Optimize data pipelines, tokenization, and curriculum design for multilingual pre-training.

Conocimientos

Distributed training

Python

English

HPC environments

Educación

PhD or Master in CS/AI or related field

Herramientas

Megatron-LM

DeepSpeed

PyTorch

NeMo

Descripción del empleo

The Barcelona Supercomputing Center (BSC-CNS) invites applications for a Senior Pre-training Engineer (RE3) to lead multilingual language model pre-training and scalable HPC workflows. You will design distributed training strategies, work with Megatron-LM, DeepSpeed and PyTorch, and collaborate with Tier-0 European supercomputing resources.

The role requires a PhD or Master in CS/AI, expert Python and DL libraries, strong English, and proven HPC experience.