Senior AI Data Engineer & Evaluation Architect

XG TECH PTE.LTD.

Singapore

On-site

SGD 90,000 - 150,000

Full time

4 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

XG Tech seeks an AI Data and Evaluation Engineer to design data pipelines and evaluation frameworks for personal AI computers, covering collection, processing, curation, and quality assessment of training data, as well as model benchmarking and performance evaluation.

You will establish reliable evaluation methodologies, identify data quality gaps, and drive continuous improvements in training datasets and model performance, ensuring AI models are evaluated accurately and optimized based on

Qualifications

  • Strong background in Data Engineering, Computer Science, Machine Learning, or a related field.
  • Proficiency in Python, with hands-on experience in data processing, storage systems, schema design, and data quality management.
  • Experience managing large-scale datasets, data validation, quality assurance, and developing reliable data processing workflows.
  • Experience designing and implementing model evaluation frameworks, benchmarks, and performance metrics to assess AI model capabilities and improvements.
  • Strong analytical and problem-solving skills, with the ability to identify data quality issues, evaluate noisy or inconsistent metrics, and establish measurable quality standards.
  • A strong academic or technical track record demonstrated through research publications, particularly in data quality, model evaluation, benchmarking, or reproducibility, as well as production data platforms, open-source contributions, or technical achievements.
  • Ability to work with internal and external stakeholders to establish data quality requirements, annotation standards, and evaluation processes

Responsibilities

  • Own collection pipelines end to end - sourcing, ingestion, storage, and versioning - for SFT, preference, and agent-trajectory data.
  • Build cleaning and quality control that is measurable: deduplication, filtering, labelling standards, spot checks, and batch acceptance criteria that a vendor can be held to.
  • Manage vendor relationships: scope the work, define quality bars, run trials, audit deliveries, and hold suppliers to the standard rather than hoping for it. Renegotiate or replace when the data does not clear the bar.
  • Build the tooling that makes quality visible - dashboards, per-batch reports, and failure taxonomies the model teams can act on rather than merely read.
  • Work with researchers so that data supply tracks what training actually needs, not what is easy to collect or what a vendor already has lying around.
  • Evaluation
  • Build evaluation baselines for the domains that matter to us now: software engineering (coding agents) and video generation.
  • On the coding side: curate and lock datasets, build reproducible end-to-end runs, and report model-level metrics that survive review by people who do not trust benchmarks by default.
  • On the video side: design a consistency-evaluation scheme, generate evaluation sets at meaningful scale, and produce model comparisons with conclusions rather than scores.
  • Build the infrastructure - test-case definition, metric plumbing, latency/token/cost accounting - so that evaluating a new model is a routine operation rather than a project.
  • Keep the loop honest: detect contamination, judge drift, and the gap between offline metrics and how the thing actually feels in use, and say so when the numbers and the experience disagree.

Skills

Data Engineering
Python proficiency
Machine Learning
Data quality management

Tools

Python

Job description

XG Tech seeks an AI Data and Evaluation Engineer to design data pipelines and evaluation frameworks for personal AI computers, covering collection, processing, curation, and quality assessment of training data, as well as model benchmarking and performance evaluation.

You will establish reliable evaluation methodologies, identify data quality gaps, and drive continuous improvements in training datasets and model performance, ensuring AI models are evaluated accurately and optimized based on

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Member of Technical Staff (AI Data)
Senior Member of Technical Staff (AI Data)

XG TECH PTE.LTD. • Singapore

On-site
SGD 90,000 - 150,000
Senior AI Model Engineer — Evaluation & Training
Senior AI Model Engineer — Evaluation & Training

Tech • Singapore

On-site
SGD 90,000 - 130,000
Flexible work
Total rewards
AI Agent Evaluation Scientist
AI Agent Evaluation Scientist

Manus AI • Singapore

On-site
SGD 90,000 - 150,000
Senior Member of Technical Staff (Inference Acceleration & Agent Framework / RSI)
Senior Member of Technical Staff (Inference Acceleration & Agent Framework / RSI)

XG TECH PTE. LTD. • Singapore

On-site
SGD 100,000 - 150,000
Agent Evaluation Engineer
Agent Evaluation Engineer

Manus AI • Singapore

On-site
SGD 90,000 - 150,000
Ai Data Engineering Lead
Ai Data Engineering Lead

optimum solutions (singapore) pte ltd • Singapore

On-site
SGD 180,000 - 260,000
Artificial Intelligence Engineer
Artificial Intelligence Engineer

Groundup.ai Pte Ltd • Singapore

On-site
SGD 140,000 - 200,000
AI Data Engineering Lead for GenAI Platforms
AI Data Engineering Lead for GenAI Platforms

optimum solutions (singapore) pte ltd • Singapore

On-site
SGD 180,000 - 260,000
Senior AI Systems Engineer — On-Device Inference & Agents
Senior AI Systems Engineer — On-Device Inference & Agents

XG TECH PTE. LTD. • Singapore

On-site
SGD 100,000 - 150,000
Data Engineer
Data Engineer

St Logistics • Singapore

On-site
SGD 120,000 - 180,000