Senior AI Researcher- Pre-training (f/m/d)

present of AI; Heidelberg; Hybrid; EUR 120,000 - 180,000; Full time

Job summary

Aleph Alpha is seeking a Senior AI Researcher focused on Pre-training, owning the training of large language models and their architectures. The role requires expertise with multi-GPU clusters and PyTorch, plus the ability to trace loss spikes and distributed bugs to actionable insights.

Ideal candidates have experience with 7B+ models, optimization math, and open frameworks like Megatron-LM, DeepSpeed or torchtitan. Collaboration across data, evaluation and hardware teams is essential.

Qualifications

• Experience pre-training large language models (7B+ parameters) on multi-GPU clusters.
• Strong optimization math understanding and practical intuition for training stability.
• Ability to diagnose loss spikes and distributed training bugs and translate results into decisions.

Responsibilities

• Own pre-training setup: optimizer, learning rate schedule, starting weights.
• Make architecture choices that affect convergence and stability of runs.
• Collaborate with data, evaluation and hardware teams to ensure reliable experiments.
• Publish research or contribute open-source work when possible.

Skills

Pre-training large language models

PyTorch expertise

Distributed training debugging

Tools

Megatron-LM

DeepSpeed

torchtitan

Job description

Senior AI Researcher- Pre-training (f/m/d)

The role centers on owning how a large language model is trained: the optimizer, learning schedule, starting weights, and architecture choices that decide whether a run converges and stays stable. A strong candidate has already pre-trained models of 7B parameters or more on clusters of 100+ GPUs, reasons comfortably about optimization math, and writes clean, tested PyTorch. They can also track down tricky failures such as loss spikes and distributed bugs, then turn noisy experiment results into clear decisions.

How it fits the AI era

The work is building the AI itself, so the job is shaped by how fast model design and training methods keep changing. Tools that help write code and run experiments can speed up routine tasks, but judgment about why a run diverged or which scaling bet is worth thousands of GPUs still rests with the researcher. Careful experiment design, honest reading of weak evidence, and teamwork across data, evaluation and hardware groups stay human.

How it builds AI experience

Someone in this role would gain hands-on experience with training runs at a scale few teams in Europe reach, plus work on models aimed at German language and regulated, high-stakes uses. To prepare, a candidate can practice with open training frameworks such as Megatron-LM, DeepSpeed or torchtitan, run small scaling-law studies, and write up what they learn on public projects.

Worth showing

• Pre-training 7B+ models at scale
• Scaling laws and hyperparameter transfer
• Debugging loss spikes and divergence
• Published research or open-source work

This is our own read, written from the employer's posting and the company's Spotlight. The employer's posting is the source of truth for the role.

About Aleph Alpha

Per the company, Aleph Alpha builds sovereign large language models and AI solutions for European enterprises and public institutions.