Senior Member of Technical Staff (AI Data)

XG TECH PTE.LTD.

Singapore

On-site

SGD 90,000 - 150,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

XG Tech seeks an AI Data and Evaluation Engineer to design data pipelines and evaluation frameworks for personal AI computers, covering collection, processing, curation, and quality assessment of training data, as well as model benchmarking and performance evaluation.

You will establish reliable evaluation methodologies, identify data quality gaps, and drive continuous improvements in training datasets and model performance, ensuring AI models are evaluated accurately and optimized based on

Qualifications

  • Strong background in Data Engineering, Computer Science, Machine Learning, or a related field.
  • Proficiency in Python, with hands-on experience in data processing, storage systems, schema design, and data quality management.
  • Experience managing large-scale datasets, data validation, quality assurance, and developing reliable data processing workflows.
  • Experience designing and implementing model evaluation frameworks, benchmarks, and performance metrics to assess AI model capabilities and improvements.
  • Strong analytical and problem-solving skills, with the ability to identify data quality issues, evaluate noisy or inconsistent metrics, and establish measurable quality standards.
  • A strong academic or technical track record demonstrated through research publications, particularly in data quality, model evaluation, benchmarking, or reproducibility, as well as production data platforms, open-source contributions, or technical achievements.
  • Ability to work with internal and external stakeholders to establish data quality requirements, annotation standards, and evaluation processes

Responsibilities

  • Own collection pipelines end to end - sourcing, ingestion, storage, and versioning - for SFT, preference, and agent-trajectory data.
  • Build cleaning and quality control that is measurable: deduplication, filtering, labelling standards, spot checks, and batch acceptance criteria that a vendor can be held to.
  • Manage vendor relationships: scope the work, define quality bars, run trials, audit deliveries, and hold suppliers to the standard rather than hoping for it. Renegotiate or replace when the data does not clear the bar.
  • Build the tooling that makes quality visible - dashboards, per-batch reports, and failure taxonomies the model teams can act on rather than merely read.
  • Work with researchers so that data supply tracks what training actually needs, not what is easy to collect or what a vendor already has lying around.
  • Evaluation
  • Build evaluation baselines for the domains that matter to us now: software engineering (coding agents) and video generation.
  • On the coding side: curate and lock datasets, build reproducible end-to-end runs, and report model-level metrics that survive review by people who do not trust benchmarks by default.
  • On the video side: design a consistency-evaluation scheme, generate evaluation sets at meaningful scale, and produce model comparisons with conclusions rather than scores.
  • Build the infrastructure - test-case definition, metric plumbing, latency/token/cost accounting - so that evaluating a new model is a routine operation rather than a project.
  • Keep the loop honest: detect contamination, judge drift, and the gap between offline metrics and how the thing actually feels in use, and say so when the numbers and the experience disagree.

Skills

Data Engineering
Python proficiency
Machine Learning
Data quality management

Tools

Python

Job description

About Company

Founded in 2022, XG Tech is driving the future of smart vehicles. Its mission is to empower the digital transformation of automobiles, moving from distributed computing to a centralized, cross-domain platform.


XG Tech focuses on the intelligent cockpit - the next frontier of differentiation - while seamlessly integrating advanced driving systems. By reimagining cars as mobile living spaces, XG Tech aligns with the evolving trend of vehicles becoming the \"third living space\".


Role Summary

As an AI Data and Evaluation Engineer, you will design and develop data pipelines and evaluation frameworks for personal AI computers, covering the collection, processing, curation, and quality assessment of training data, as well as model benchmarking and performance evaluation. You will establish reliable evaluation methodologies to measure model capabilities, identify data quality gaps, and drive continuous improvements in training datasets and model performance, ensuring AI models are evaluated accurately and optimized based on measurable results.


Key Responsibilities

Data supply and quality



  • Own collection pipelines end to end - sourcing, ingestion, storage, and versioning - for SFT, preference, and agent-trajectory data.

  • Build cleaning and quality control that is measurable: deduplication, filtering, labelling standards, spot checks, and batch acceptance criteria that a vendor can be held to.

  • Manage vendor relationships: scope the work, define quality bars, run trials, audit deliveries, and hold suppliers to the standard rather than hoping for it. Renegotiate or replace when the data does not clear the bar.

  • Build the tooling that makes quality visible - dashboards, per-batch reports, and failure taxonomies the model teams can act on rather than merely read.

  • Work with researchers so that data supply tracks what training actually needs, not what is easy to collect or what a vendor already has lying around.


Evaluation



  • Build evaluation baselines for the domains that matter to us now: software engineering (coding agents) and video generation.

  • On the coding side: curate and lock datasets, build reproducible end-to-end runs, and report model-level metrics that survive review by people who do not trust benchmarks by default.

  • On the video side: design a consistency-evaluation scheme, generate evaluation sets at meaningful scale, and produce model comparisons with conclusions rather than scores.

  • Build the infrastructure - test-case definition, metric plumbing, latency/token/cost accounting - so that evaluating a new model is a routine operation rather than a project.

  • Keep the loop honest: detect contamination, judge drift, and the gap between offline metrics and how the thing actually feels in use, and say so when the numbers and the experience disagree.


How you will stand out


  • Strong background in Data Engineering, Computer Science, Machine Learning, or a related field, with solid experience building and maintaining scalable data pipelines and infrastructure.

  • Proficiency in Python, with hands-on experience in data processing, storage systems, schema design, and data quality management.

  • Demonstrated experience managing large-scale datasets, data validation, quality assurance, and developing reliable data processing workflows.

  • Experience designing and implementing model evaluation frameworks, benchmarks, and performance metrics to assess AI model capabilities and improvements.

  • Strong analytical and problem-solving skills, with the ability to identify data quality issues, evaluate noisy or inconsistent metrics, and establish measurable quality standards.

  • A strong academic or technical track record demonstrated through research publications, particularly in data quality, model evaluation, benchmarking, or reproducibility, as well as production data platforms, open-source contributions, or technical achievements.

  • Ability to work with internal and external stakeholders to establish data quality requirements, annotation standards, and evaluation processes


Nice to have:


  • Experience working with AI agent trajectories, UI interaction data, or multimodal datasets.

  • Familiarity with data annotation platforms and human-in-the-loop quality assurance processes.

  • Experience with LLM-as-a-Judge evaluation, calibration, and its limitations.

  • Knowledge of privacy-aware data processing and secure handling of personal or sensitive data.

  • Experience optimizing storage, computing, and processing costs for large-scale data infrastructure.


XG Tech is committed to providing equal employment opportunities by country, state, and local laws. XG Tech does not discriminate against employees or applicants based on conditions such as race, color, gender identity and/or expression, sexual orientation, marital and/or parental status, religion, political opinion, nationality, ethnic background or social origin, social status, disability, age, indigenous status, and union.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Member of Technical Staff (Inference Acceleration & Agent Framework / RSI)
Senior Member of Technical Staff (Inference Acceleration & Agent Framework / RSI)

XG TECH PTE. LTD. • Singapore

On-site
SGD 100,000 - 150,000
Senior Member of Technical Staff (Embedded Development, On-Device Integration)
Senior Member of Technical Staff (Embedded Development, On-Device Integration)

XG TECH PTE.LTD. • Singapore

On-site
SGD 120,000 - 190,000
Senior Member of Technical Staff (Post-Training)
Senior Member of Technical Staff (Post-Training)

XG TECH PTE. LTD. • Singapore

On-site
SGD 120,000 - 180,000
Senior AI Data Engineer & Evaluation Architect
Senior AI Data Engineer & Evaluation Architect

XG TECH PTE.LTD. • Singapore

On-site
SGD 90,000 - 150,000
Senior Member of Technical Staff (Video Generation & 3D, Cockpit Themes)
Senior Member of Technical Staff (Video Generation & 3D, Cockpit Themes)

XG TECH PTE. LTD. • Singapore

On-site
SGD 120,000 - 180,000
Senior Member of Technical Staff (Model Routing)
Senior Member of Technical Staff (Model Routing)

XG TECH PTE.LTD. • Singapore

On-site
SGD 120,000 - 180,000
Image Algorithm Engineer
Image Algorithm Engineer

Acrab • Singapore

On-site
SGD 120,000 - 180,000
Senior AI Engineer - Agentic AI Evaluation
Senior AI Engineer - Agentic AI Evaluation

RESARO SINGAPORE PTE. LTD. • Singapore

On-site
SGD 120,000 - 180,000
AI Engineer
AI Engineer

DELTA SG CONSULTING • Singapore

On-site
SGD 120,000 - 180,000
Software Engineer - AI
Software Engineer - AI

Anchor Search Group Pte Ltd • Singapore

On-site
SGD 90,000 - 150,000