Applied AI Evaluations Intern: Benchmark LLMs & Data Pipelines

NUSX; Singapore; On-site; SGD 60,000 - 100,000; Full time

Job summary

NUSX Labs is the technical arm of NUSX, turning research into inventions and licensed deals. As a hands-on team, you will build and benchmark datasets for multiple document types, and contribute to an LLM pipeline that reads patent documents into structured data and a query layer for analytics.

You will supervise by an engineer, work on ground-truth datasets, evaluate model outputs, and prepare quality-acceptance reports used to sign off deliveries.

Qualifications

• Experience building and shipping a data pipeline to completion.
• Proficiency in Python for processing structured data (JSON/CSV).
• Familiarity with evaluation concepts (precision, recall, accuracy) and LM evaluation metrics.
• Willingness to learn domain knowledge (e.g. patents).
• Familiarity with version control (Git).
• Willingness to use AI tools (Claude, Antigravity) daily.
• Nice to have: basic SQL; basic cloud experience (GCP/AWS); familiarity with data engineering concepts; comfortable reading and working in English and Chinese.

Responsibilities

• Build benchmark datasets used to evaluate model performance, across approximately 23 document types, using Claude and Antigravity to draft and an annotation tool to review and refine
• Run the blind-test evaluation matrix on vendor deliverables and measure extraction reliability against the ground truth
• Identify edge cases, structural mismatches, and hallucinations that surface-level review would miss, and document them
• Review existing system interfaces to determine what each screen stores, and build the gold field and schema catalog
• Compile findings into quality-acceptance reports used by the team to sign off on deliveries
• Scope may extend beyond the responsibilities listed above

Skills

Python programming

Data pipelines

LLM evaluation

English & Chinese literacy

Git/version control

SQL (basic)

Tools

Claude

Antigravity

GCP/AWS

Job description

NUSX Labs is the technical arm of NUSX, turning research into inventions and licensed deals. As a hands-on team, you will build and benchmark datasets for multiple document types, and contribute to an LLM pipeline that reads patent documents into structured data and a query layer for analytics.

You will supervise by an engineer, work on ground-truth datasets, evaluate model outputs, and prepare quality-acceptance reports used to sign off deliveries.