Lead AI Data Engineer, LLM Training Pipelines

Kuailu Software; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Kuailu Software is seeking a data engineer to build and maintain end-to-end training data pipelines for Large Language Models (LLMs), covering data cleaning, deduplication, format conversion, quality filtering, data mixture, and version management.

You will collaborate with engineers to convert ReAct trajectories and Function Calling workflows into training-ready data formats, while overseeing data versioning and traceability for each iteration.

Qualifications

• Bachelor's degree or higher in CS/AI/Data Science or related field.
• Proficient in Python with hands-on experience processing large-scale text datasets.
• Experience building LLM SFT/RLHF data pipelines and data quality controls.
• Familiar with ChatML/Function Calling formats and multi-turn data structures.

Responsibilities

• Build end-to-end training data pipelines for LLMs across data cleaning, deduplication, format conversion, and quality filtering.
• Design and adjust data mix between SFT training data and Replay data.
• Manage prompt pool for distillation training, with deduplication and sampling across domains.
• Establish data quality assurance with automated and manual sampling-based QC.
• Collaborate to convert ReAct trajectories and Function Calling workflows into training-ready formats.
• Manage training data versioning and traceability for each training iteration.

Skills

Python

pandas

Data processing

LLM SFT/RLHF pipelines

Data deduplication

Quality filtering

ChatML formats

Function Calling data formats

Data versioning

Education

Bachelor's degree in Computer Science / AI / Data Science or related field

Tools

ChatML

Function Calling formats

Job description

Kuailu Software is seeking a data engineer to build and maintain end-to-end training data pipelines for Large Language Models (LLMs), covering data cleaning, deduplication, format conversion, quality filtering, data mixture, and version management.

You will collaborate with engineers to convert ReAct trajectories and Function Calling workflows into training-ready data formats, while overseeing data versioning and traceability for each iteration.