Senior LLM Pre-Training Engineer

UMI TECH; Singapore; On-site; SGD 150,000 - 230,000; Full time

Job summary

UMI TECH in Singapore seeks an expert to lead LLM pre-training initiatives, focusing on strategy, architecture, and large-scale distributed training. You will oversee long-context training, monitoring, and optimization of training runs, delivering robust models and scalable data pipelines.

The role requires hands-on experience with 7B+ parameter models, Megatron-LM/DeepSpeed/FSDP, and 64+ GPU environments, plus a track record of publications or infrastructure work in top AI venues.

Qualifications

• Master's degree or above in Computer Science, Artificial Intelligence, NLP, Machine Learning, Distributed Systems, or a related field.
• Hands-on experience with complete LLM pre-training projects.
• Experience participating in the pre-training of 7B+ parameter models.
• Strong expertise in large-scale distributed training frameworks such as Megatron-LM, DeepSpeed, or FSDP.
• Practical experience with 64+ GPU training environments.
• Solid understanding of large-scale pre-training data pipelines, including data cleaning, deduplication, quality filtering, tokenization, data mixing, and data quality optimization.
• Strong ability to analyze training loss, gradients, convergence, and training stability.
• Experience troubleshooting large-scale distributed training issues.
• Familiarity with long-context training and extension techniques, including RoPE scaling, NTK-aware interpolation, and YaRN.
• Experience with 70B+ parameter model pre-training; MoE models; publications in top AI/ML conferences; training infrastructure optimization.

Responsibilities

• Pre-training strategy & Architecture for large language models.
• Long-context training, extension techniques, and monitoring.
• Training monitoring, optimization, and stability analysis.
• Evaluation and iterative optimization of training methods.

Skills

LLM pre-training

Distributed training

Long-context training

Model evaluation

Training optimization

Education

Master's degree or higher in CS/AI/ML

Tools

Megatron-LM

DeepSpeed

FSDP

Job description

UMI TECH in Singapore seeks an expert to lead LLM pre-training initiatives, focusing on strategy, architecture, and large-scale distributed training. You will oversee long-context training, monitoring, and optimization of training runs, delivering robust models and scalable data pipelines.

The role requires hands-on experience with 7B+ parameter models, Megatron-LM/DeepSpeed/FSDP, and 64+ GPU environments, plus a track record of publications or infrastructure work in top AI venues.