PhD Position “Foundation Models & Efficient LLM Training” – DS-NLP Lab @HSG (m/f/d)

University of St.Gallen; Schweiz; Vor Ort; CHF 60.000 - 90.000; Vollzeit

Benefits dieser Stelle

Fully funded PhD position

Modern compute infrastructure

Support for publishing at top-tier-con

Zusammenfassung

The University of St.Gallen (HSG) in Switzerland invites applications for a fully funded PhD position focused on foundation model training and efficient learning methods. You will pursue a doctoral thesis while researching training dynamics of large language models and incorporating linguistic insights into token weighting strategies.

The role includes teaching (~2 SWS per semester), supervision of Bachelor and Master theses, and supporting the chair in organizational and research tasks.

Qualifikationen

• Master's degree in computer science, ML, computational linguistics or related field.
• Desirable: prior research experience or publications in NLP, ML, or related areas.
• Strong programming skills in Python.
• Desirable: experience with PyTorch or TensorFlow/Keras.
• Experience or strong interest in training large language models (transformers).
• Experience with efficient training techniques (e.g., LoRA, quantization, sparsity, distillation).
• Knowledge of scaling laws, training dynamics, or evaluation of generative models.
• Experience with dataset construction and large-scale data processing.
• Desirable: background or interest in linguistics or computational linguistics, particularly tokenization and token weighting strategies.
• Ability to work independently, structure research, and manage time.
• Motivation to contribute to teaching and collaborative research.

Aufgaben

• You will work on your doctoral thesis in the area of foundation model training and efficient learning methods.
• You design and conduct research on training dynamics of large language models, including scaling behavior, optimization, and generalization.
• You incorporate linguistic insights into the development and analysis of language models, with a particular focus on token weighting strategies and their effects on training dynamics.
• You engage in teaching activities (approx. 2 SWS per semester), including tutorials, exercises, and supervision of Bachelor's and Master's theses and projects.
• You support the chair in organizational and research-related tasks.

Kenntnisse

Strong research mindset

English writing

Ausbildung

Master's degree in computer science / ML / computational linguistics or related field

Tools

Python

PyTorch

TensorFlow/Keras

Jobbeschreibung

• You will work on your doctoral thesis in the area of foundation model training and efficient learning methods
• You design and conduct research on training dynamics of large language models, including scaling behavior, optimization, and generalization
• You incorporate linguistic insights into the development and analysis of language models, with a particular focus on token weighting strategies and their effects on training dynamics
• You engage in teaching activities (approx. 2 SWS per semester), including tutorials, exercises, and supervision of Bachelor's and Master's theses and projects
• You support the chair in organizational and research-related tasks

Your tasks

• You will work on your doctoral thesis in the area of foundation model training and efficient learning methods
• You design and conduct research on training dynamics of large language models, including scaling behavior, optimization, and generalization
• You incorporate linguistic insights into the development and analysis of language models, with a particular focus on token weighting strategies and their effects on training dynamics
• You engage in teaching activities (approx. 2 SWS per semester), including tutorials, exercises, and supervision of Bachelor's and Master's theses and projects
• You support the chair in organizational and research-related tasks

Your profile

• You hold a master's degree in computer science, machine learning, computational linguistics, or a related field
• Desirable: prior research experience or publications in NLP, ML, or related areas
• You have strong programming skills in Python
• Desirable: experience with PyTorch or TensorFlow/Keras
• You have experience or strong interest in:
• Training large language models (transformers)
• Efficient training techniques (e.g., LoRA, quantization, sparsity, distillation)
• Scaling laws, training dynamics, or evaluation of generative models
• Dataset construction and large-scale data processing
• Desirable:
• Background or interest in linguistics or computational linguistics, particularly in relation to tokenization and token weighting strategies
• You are able to work independently, structure your research, and manage your time effectively
• You are motivated to contribute to teaching and collaborative research

We offer

• fully funded PhD position in a dynamic and fast-growing research area
• Access to modern compute infrastructure
• Support for publishing at top-tier conferences
• A collaborative and international research environment
• Full supervision and mentoring throughout your PhD journey

Inspirational Work Environment

At the HSG, as Europe's leading business university, we operate in an entrepreneurial environment where you will meet highly qualified, ambitious experts - both regionally and internationally. For example, you benefit from a modern IT infrastructure and, in our library via HSGswisscovery, from nationwide access to the wealth of scientific information available.

Executive Education

We support internal and external executive education opportunities. Our internal executive education programme offers you a practical, dedicated and didactically mature transfer of knowledge in the form of seminars, e-learning, coaching and collegial advice. We support task-specific external training and executive education financially as well as with regard to the time required.

Network

We promote our culture of cross-functional collaboration and create space for forging valuable relationships - also with external partners. Regular exchanges and internal events help to strengthen these relationships and make them long-lasting.

Family & Career

We provide our employees with flexible work time models with a variable number of hours per week. As a family-friendly employer, we offer part-time positions, 16 weeks of maternity leave and a subsidised crèche, as well as support with childcare during the vacations.

Equal Opportunities

The University enables equal opportunities for all employees and students. This applies to degree courses and work, as well as academic careers, university management positions and the reconciliation of studies, work and private commitments. We are committed to this with various measures in the areas of teaching, research, administration and the Presidency.

Fit and Healthy

We help you stay fit and healthy. In addition to our diverse range of sports activities at the university, most of which are free of charge, and campaign weeks for corporate health management, we also offer you fruit in the workplace. In our canteen you will find a wide selection of freshly prepared meals - from vegetarian to fish dishes - at discounted prices.

Further information on working at the HSG can be found here.

University of St.Gallen

«A place where knowledge is created» - As one of Europe's leading universities of economics and business administration, the University of St.Gallen (HSG), Switzerland, is committed to the education of over 10'000 students. The HSG is one of the largest employers in the region and provides an attractive and innovative environment for more than 3'800 researchers, educators and professional staff.