Senior AI Researcher- Pre-training (f/m/d)

FDE Times

Heidelberg

On-site

EUR 120,000 - 180,000

Full time

6 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

30 days vacation
Wellhub fitness
Nilo health
Pension plan
Transit ticket
Tech equipment budget
Hybrid work
Bike lease

Job summary

Aleph Alpha in Heidelberg is hiring a Senior AI Researcher for our Pre-training team to advance architecture, optimization, stability, and scaling of next-generation foundation models. You will translate mathematical reasoning into principled training decisions, from small proxy experiments to multi-thousand-GPU runs.

We seek an expert who can merge rigorous experimental design with production-quality code, directly influencing model quality, run reliability, and efficiency across the

Qualifications

  • Strong Python and PyTorch experience for training workflows.
  • Proven track record in ML research or shipped models.
  • Background in optimization, scaling, and distributed training.
  • Understanding of transformer training dynamics and large-scale runs.
  • Ability to run rigorous experiments and translate results into production code.
  • Experience pre-training large models on substantial infrastructure.

Responsibilities

  • Own core training recipe elements and PyTorch architecture improvements.
  • Develop scaling laws and proxy experiments to predict multi-thousand-GPU behavior.
  • Investigate convergence issues and resolve distributed system failures.
  • Coordinate with Compute, Data, Evaluation, and Post-Training teams.

Skills

Python
PyTorch
LLM training
Distributed training
Experiment design
Software engineering
Research publication track
Mob programming

Tools

Megatron-LM
DeepSpeed
torchtitan
NVIDIA GPU clusters

Job description

Our Mission

Aleph Alpha is one of the few companies in Europe doing serious foundation model pre-training like our recently released Kolibri-1 model. Our customers - in finance, manufacturing, public administration - need models that understand German, meet European regulatory requirements, and work reliably in high-stakes settings. We're building that in Heidelberg.

We are hiring a Senior AI Researcher to join our Pre-training team and to advance the architecture and training of our next generation of foundation models. If you are excited about designing inference-efficient architectures, optimizing training recipes that scale reliably, and training models on a large scale cluster (thousands of NVIDIA Blackwell GPUs), we would love to hear from you.

Team Culture

We foster a culture built on ownership, autonomy, and empowerment. Teams and individual contributors are trusted to take responsibility for their work and drive meaningful impact. We maintain a flat organisational structure with efficient, supportive management that enables quick decision-making, open communication, and a strong sense of shared purpose. We collaborate closely on complex technical problems, working in pairs or using mob programming to resolve challenging issues.

About the Role

As a Senior AI Researcher in Pre-training (f/m/d), you will own the critical technical levers that determine the success of our next-generation models: architecture, optimization, stability, and scaling.

Working at the high-leverage intersection of research and engineering, you will translate mathematical reasoning and empirical observations into principled training decisions - from small-scale proxy experiments to multi-thousand-GPU runs.

We are looking for an expert who can combine rigorous experimental design with high-quality production code, directly influencing model quality, run reliability, and the efficiency of the models we ship.

Your Responsibilities
  • Recipe & Architecture Optimization: Own core elements of the training recipe (optimizers, schedules, initialization) and design PyTorch-based architectural improvements to maximize convergence, stability, and training efficiency.
  • Scaling Strategy & Predictability: Develop hyperparameter scaling laws and scale-up methodologies, using small-scale proxy experiments to reliably predict multi-thousand-GPU behavior and de-risk major training decisions.
  • Stability, Diagnostics & Debugging: Investigate complex convergence issues (loss spikes, divergence) and resolve hard-to-reproduce distributed system failures like communication bottlenecks, race conditions, and synchronization errors.
  • System-Model Co-Design: Partner with Compute Performance, Data, Evaluation, and Post-Training teams to align the model lifecycle with hardware constraints, memory bandwidth, and communication topologies.
Core Qualifications
  • You are proficient in Python and deeply familiar with PyTorch-based training workflows.
  • You have a strong track record in machine learning research and software engineering, demonstrated through shipped models, impactful open-source contributions, or published research.
  • You have a strong mathematical foundation and are comfortable reasoning formally about optimisation, scaling behaviour, and training dynamics.
  • You deeply understand transformer training dynamics, optimisation, and the behaviour of large distributed training jobs.
  • You can design rigorous experiments, reason clearly from noisy results, and translate empirical observations into robust training decisions.
  • Hands-on experience pre-training large models (e.g., 7B+ parameters) on substantial infrastructure (e.g., 100+ GPU clusters).
  • You apply strong software engineering practices, including writing maintainable, well-tested code and supporting reproducible experimentation workflows.
  • You are able to implement complex model architectures efficiently and reliably and to debug complex issues across model code, training dynamics, and distributed systems.
  • You collaborate effectively within a research and engineering team and communicate clearly about your work across Pre-training and the broader AAR/AA organization.
  • You are able to work in Germany and collaborate regularly on site in Heidelberg as part of the Pre-training team.
Preferred Qualifications
  • Large-Scale Training: Hands-on experience training LLMs or multimodal models on large GPU clusters using distributed frameworks (e.g., Megatron-LM, DeepSpeed, torchtitan).
  • Predictive Scaling: Familiarity with scaling laws, hyperparameter transfer, or methods for predicting large-scale training behavior from smaller proxy runs.
  • Stability & Performance: Experience profiling distributed jobs and diagnosing training anomalies like loss spikes, numerical instability, or optimizer pathologies.
  • Advanced Architectures: Exposure to sparse training approaches (e.g., Mixture-of-Experts) and an understanding of their routing and systems trade-offs.
  • Track Record of Impact: Demonstrated research excellence through top-tier publications (NeurIPS, ICML, ICLR), impactful open-source contributions, or significant shipped technical work.
  • Systems Curiosity: Low-level kernel optimization is not required, but we highly value a strong curiosity about the hardware and systems constraints that shape scale.
What we offer
  • Become part of an AI revolution!
  • 30 days of paid vacation
  • Access to a variety of fitness & wellness offerings via Wellhub
  • Mental health support through nilo.health
  • Substantially subsidized company pension plan for your future security
  • Subsidized Germany-wide transportation ticket
  • Budget for additional technical equipment
  • Flexible working hours for better work-life balance and hybrid working model
  • JobRad Bike Lease
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI Researcher- Pre-training (f/m/d)
Senior AI Researcher- Pre-training (f/m/d)

Aleph Alpha • Heidelberg

Hybrid
EUR 120,000 - 180,000
30 days vacation
Wellhub fitness
nilo.health
+5
AI Researcher - Pre-training (f/m/d)
AI Researcher - Pre-training (f/m/d)

alephalpha • Germany

On-site
EUR 90,000 - 130,000
AI Researcher - Pre-training (f/m/d)
AI Researcher - Pre-training (f/m/d)

Alephalpha • Heidelberg

On-site
EUR 90,000 - 140,000
Senior AI R&D Engineer - Performance - Pre-training (f/m/d)
Senior AI R&D Engineer - Performance - Pre-training (f/m/d)

FDE Times • Heidelberg

On-site
EUR 90,000 - 150,000
30 days vacation
Wellhub fitness access
nilo.health support
+6
AI Researcher - Pre-training (f/m/d)
AI Researcher - Pre-training (f/m/d)

Alephalpha • Berlin

On-site
EUR 90,000 - 130,000
AI Researcher - Pre-training (f/m/d)
AI Researcher - Pre-training (f/m/d)

FDE Times • Berlin

On-site
EUR 90,000 - 150,000
Health insurance
AI Researcher - Pre-training (f/m/d)
AI Researcher - Pre-training (f/m/d)

DeepCamp • Heidelberg

Hybrid
EUR 90,000 - 150,000
Senior AI R&D Engineer - Performance - Pre-training (f/m/d)
Senior AI R&D Engineer - Performance - Pre-training (f/m/d)

Aleph Alpha GmbH • Heidelberg

On-site
EUR 90,000 - 130,000
30 days vacation
Wellhub fitness & wellness offerings
Mental health support via nilo.health
+5
Senior AI R&D Engineer – Model Training (f/m/d)
Senior AI R&D Engineer – Model Training (f/m/d)

Aleph Alpha • Heidelberg

Hybrid
EUR 90,000 - 130,000
30 days vacation
Wellhub fitness access
Mental health support
+6
Senior AI R&D Engineer – Model Training (f/m/d)
Senior AI R&D Engineer – Model Training (f/m/d)

Aleph Alpha • Berlin

Hybrid
EUR 90,000 - 140,000
30 days vacation
Wellhub fitness offerings
Mental health support
+5