Senior Pre-training Engineer - Multilingual LLMs on HPC

Barcelona Supercomputing Center; Barcelona; Presencial; EUR 80.000 - 110.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Restaurant tickets

Private health insurance

Relocation support

Descripción de la vacante

The Barcelona Supercomputing Center invites applications for a Senior Pre-training Engineer (RE3). Based in Barcelona, you will lead the design and implementation of large-scale multilingual pre-training pipelines on multi-node HPC systems, ensuring efficient scaling across hundreds of GPUs.

You will work with state-of-the-art frameworks (Megatron-LM, DeepSpeed, PyTorch), mentor junior team members, and collaborate with infrastructure teams to maximize throughput.

Formación

• PhD or Master’s degree in Computer Science, AI or related field.
• Extensive experience in pre-training large language models and multilingual pipelines.
• Proven expertise with distributed training frameworks (Megatron-LM, DeepSpeed, PyTorch FSDP or NeMo).

Responsabilidades

• Lead the technical design of large-scale multilingual pre-training pipelines on multi-node HPC infrastructure.
• Design distributed training strategies to scale across hundreds to thousands of GPUs.
• Research novel model architectures like Mixture-of-Experts for pre-training.
• Collaborate with HPC teams to maximize cluster utilization and reliability.

Conocimientos

Distributed training

Megatron-LM

DeepSpeed

PyTorch

Python

English

Educación

PhD or Master’s in CS/AI

Herramientas

Megatron-LM

DeepSpeed

PyTorch FSDP

NeMo

Descripción del empleo

The Barcelona Supercomputing Center invites applications for a Senior Pre-training Engineer (RE3). Based in Barcelona, you will lead the design and implementation of large-scale multilingual pre-training pipelines on multi-node HPC systems, ensuring efficient scaling across hundreds of GPUs.

You will work with state-of-the-art frameworks (Megatron-LM, DeepSpeed, PyTorch), mentor junior team members, and collaborate with infrastructure teams to maximize throughput.