Artificial Intelligence Engineer (Pre/post training)

UMI TECH; Singapore; On-site; SGD 150,000 - 230,000; Full time

Job summary

UMI TECH in Singapore seeks an expert to lead LLM pre-training initiatives, focusing on strategy, architecture, and large-scale distributed training. You will oversee long-context training, monitoring, and optimization of training runs, delivering robust models and scalable data pipelines.

The role requires hands-on experience with 7B+ parameter models, Megatron-LM/DeepSpeed/FSDP, and 64+ GPU environments, plus a track record of publications or infrastructure work in top AI venues.

Qualifications

• Master's degree or above in Computer Science, Artificial Intelligence, NLP, Machine Learning, Distributed Systems, or a related field.
• Hands-on experience with complete LLM pre-training projects.
• Experience participating in the pre-training of 7B+ parameter models.
• Strong expertise in large-scale distributed training frameworks such as Megatron-LM, DeepSpeed, or FSDP.
• Practical experience with 64+ GPU training environments.
• Solid understanding of large-scale pre-training data pipelines, including data cleaning, deduplication, quality filtering, tokenization, data mixing, and data quality optimization.
• Strong ability to analyze training loss, gradients, convergence, and training stability.
• Experience troubleshooting large-scale distributed training issues.
• Familiarity with long-context training and extension techniques, including RoPE scaling, NTK-aware interpolation, and YaRN.
• Experience with 70B+ parameter model pre-training; MoE models; publications in top AI/ML conferences; training infrastructure optimization.

Responsibilities

• Pre-training strategy & Architecture for large language models.
• Long-context training, extension techniques, and monitoring.
• Training monitoring, optimization, and stability analysis.
• Evaluation and iterative optimization of training methods.

Skills

LLM pre-training

Distributed training

Long-context training

Model evaluation

Training optimization

Education

Master's degree or higher in CS/AI/ML

Tools

Megatron-LM

DeepSpeed

FSDP

Job description

Key Skills

Job Responsibilities

• Pre-training Strategy & Architecture
• Long-context Training
• Training Monitoring & Optimization
• Evaluation & Iterative Optimization

Job Requirements

Education

Master's degree or above in Computer Science, Artificial Intelligence, NLP, Machine Learning, Distributed Systems, or a related field.

LLM Pre-training Experience

Hands-on experience with complete LLM pre-training projects.

Experience participating in the pre-training of 7B+ parameter models.

Distributed Training

Strong expertise in large-scale distributed training frameworks such as Megatron-LM, DeepSpeed, or FSDP.

Practical experience with 64+ GPU training environments.

Solid understanding of large-scale pre-training data pipelines, including data cleaning, deduplication, quality filtering, tokenization, data mixing, and data quality optimization.

• Strong ability to analyze training loss, gradients, convergence, and training stability.
• Experience troubleshooting large-scale distributed training issues.

Long-context Training

• Familiarity with long-context training and extension techniques, including RoPE scaling, NTK-aware interpolation, and YaRN.

Preferred Qualifications

• Experience with 70B+ parameter model pre-training.
• Experience with MoE (Mixture-of-Experts) model pre-training.
• Publications in top-tier AI/ML conferences such as NeurIPS, ICML, ICLR, ACL, or EMNLP, particularly in LLM pre-training, model architecture, or training optimization.
• Experience optimizing large-scale GPU clusters and training infrastructure.