LLM Pre-Training Engineer — Scale & Optimize

UMELIFE (SINGAPORE) PTE. LTD.; Singapore; On-site; SGD 180,000 - 260,000; Full time

Job summary

UMELIFE (SINGAPORE) PTE. LTD. is seeking an expert in large-scale LLM pre-training to drive strategy, data pipelines, and distributed training across multi-GPU clusters.

The role requires hands-on experience with 7B+ parameter models, Megatron-LM, and DeepSpeed, with a focus on training stability and optimization. The candidate will work on RoPE scaling, NTK-aware interpolation, and YaRN-related techniques to extend long-context capabilities and improve efficiency in a high-performance

Qualifications

• Bachelor's degree or above in CS, AI, NLP, ML, distributed systems or related field.
• LLM pre-training experience and hands-on experience with complete LLM pre-training projects.
• Experience pre-training 7B+ parameter models.
• Experience with large-scale distributed training frameworks.
• 64+ GPU training environments experience.
• Solid understanding of large-scale pre-training data pipelines.
• Experience fine-tuning and optimization for long-context models.
• Publications in top AI/ML conferences are a plus.

Responsibilities

• Lead pre-training strategy and architecture for large-scale models.
• Handle pre-training data engineering and pipeline quality.
• Coordinate distributed training across 64+ GPUs.
• Monitor training, debug issues, and optimize convergence.
• Evaluate models and iteratively optimize training performance.
• Work on long-context training techniques and extensions.

Skills

LLM pre-training

Large-scale distributed training

Data engineering concepts

Training monitoring & debugging

Model evaluation & optimization

Troubleshooting distributed training

Long-context training

RoPE scaling awareness

NTK-aware interpolation

YaRN technique knowledge

Education

Bachelor's degree or above in Computer Science, AI, NLP, ML, Distributed Systems or related field

Tools

Megatron-LM

DeepSpeed

FSDP

RoPE scaling

NTK-aware interpolation

YaRN

Job description

UMELIFE (SINGAPORE) PTE. LTD. is seeking an expert in large-scale LLM pre-training to drive strategy, data pipelines, and distributed training across multi-GPU clusters.

The role requires hands-on experience with 7B+ parameter models, Megatron-LM, and DeepSpeed, with a focus on training stability and optimization. The candidate will work on RoPE scaling, NTK-aware interpolation, and YaRN-related techniques to extend long-context capabilities and improve efficiency in a high-performance