PhD Position “Foundation Models & Efficient LLM Training” – DS-NLP Lab @HSG (m/f/d)

Universität St.Gallen; Sankt Gallen; Vor Ort; CHF 48.000 - 64.000; Vollzeit

Zusammenfassung

Universität St.Gallen invites applications for a fully funded PhD position focusing on foundation model training and efficient learning methods. You will pursue your doctoral thesis under supervision in a collaborative, international research environment.

Responsibilities include research on training dynamics of large language models, integrating linguistic insights, teaching duties, and supporting organizational tasks at the chair.

Qualifikationen

• Master's degree in CS, ML, computational linguistics, or related field.
• Strong Python programming skills.
• Desirable: publications in NLP/ML.

Aufgaben

• Work on your doctoral thesis in foundation model training and efficient learning methods.
• Design and conduct research on training dynamics of large language models, incl. scaling, optimization, generalization.
• Incorporate linguistic insights into language model development and analysis, focusing on token weighting strategies.
• Engage in teaching activities (≈2 SWS/semester) including tutorials, exercises, supervision of theses, and projects.
• Support the chair in organizational and research-related tasks.

Kenntnisse

Python

PyTorch

NLP basics

Ausbildung

Master's degree in CS/ML/Computational Linguistics

Tools

TensorFlow/Keras

Jobbeschreibung

We offer
fully funded PhD position in a dynamic and fast-growing research area
Access to modern compute infrastructure
Support for publishing at top-tier conferences
A collaborative and international research environment
Full supervision and mentoring throughout your PhD journey

Your tasks

• You will work on your doctoral thesis in the area of foundation model training and efficient learning methods
• You design and conduct research on training dynamics of large language models, including scaling behavior, optimization, and generalization
• You incorporate linguistic insights into the development and analysis of language models, with a particular focus on token weighting strategies and their effects on training dynamics
• You engage in teaching activities (approx. 2 SWS per semester), including tutorials, exercises, and supervision of Bachelor's and Master's theses and projects
• You support the chair in organizational and research-related tasks

Your profile

• You hold a master's degree in computer science, machine learning, computational linguistics, or a related field
• Desirable: prior research experience or publications in NLP, ML, or related areas
• You have strong programming skills in Python
• Desirable: experience with PyTorch or TensorFlow/Keras
• You have experience or strong interest in:
• Training large language models (transformers)
• Efficient training techniques (e.g., LoRA, quantization, sparsity, distillation)
• Scaling laws, training dynamics, or evaluation of generative models
• Dataset construction and large-scale data processing
• Desirable:
• Background or interest in linguistics or computational linguistics, particularly in relation to tokenization and token weighting strategies
• You are able to work independently, structure your research, and manage your time effectively
• You are motivated to contribute to teaching and collaborative research

\"A place where knowledge is created\" - As one of Europe's leading universities of economics and business administration, the University of St.Gallen (HSG), Switzerland, is committed to the education of over 10'000 students. The HSG is one of the largest employers in the region and provides an attractive and innovative environment for more than 3'800 researchers, educators and professional staff.