Applied AI Engineer, Evaluations Intern

NUSX; Singapore; On-site; SGD 60,000 - 100,000; Full time

Job summary

NUSX Labs is the technical arm of NUSX, turning research into inventions and licensed deals. As a hands-on team, you will build and benchmark datasets for multiple document types, and contribute to an LLM pipeline that reads patent documents into structured data and a query layer for analytics.

You will supervise by an engineer, work on ground-truth datasets, evaluate model outputs, and prepare quality-acceptance reports used to sign off deliveries.

Qualifications

• Experience building and shipping a data pipeline to completion.
• Proficiency in Python for processing structured data (JSON/CSV).
• Familiarity with evaluation concepts (precision, recall, accuracy) and LM evaluation metrics.
• Willingness to learn domain knowledge (e.g. patents).
• Familiarity with version control (Git).
• Willingness to use AI tools (Claude, Antigravity) daily.
• Nice to have: basic SQL; basic cloud experience (GCP/AWS); familiarity with data engineering concepts; comfortable reading and working in English and Chinese.

Responsibilities

• Build benchmark datasets used to evaluate model performance, across approximately 23 document types, using Claude and Antigravity to draft and an annotation tool to review and refine
• Run the blind-test evaluation matrix on vendor deliverables and measure extraction reliability against the ground truth
• Identify edge cases, structural mismatches, and hallucinations that surface-level review would miss, and document them
• Review existing system interfaces to determine what each screen stores, and build the gold field and schema catalog
• Compile findings into quality-acceptance reports used by the team to sign off on deliveries
• Scope may extend beyond the responsibilities listed above

Skills

Python programming

Data pipelines

LLM evaluation

English & Chinese literacy

Git/version control

SQL (basic)

Tools

Claude

Antigravity

GCP/AWS

Job description

NUSX Labs is NUSX's technical arm, turning NUS research into inventions, patents, and licensed deals. We're a small, hands-on team, and every AI system we ship is graded against a ground-truth dataset built by hand: for six months, you're the one building it, and your judgments become the benchmark the next model is measured against. It's judgment work, not data entry, across an LLM pipeline that reads patent documents into structured data, a natural-language query layer, and early-signal deal-sourcing tools. You'll be supervised directly by an engineer on the team, with real stakeholder-facing projects and deployment experience.

Roles And Responsibilities

• Build benchmark datasets used to evaluate model performance, across approximately 23 document types, using Claude and Antigravity to draft and an annotation tool to review and refine
• Run the blind-test evaluation matrix on vendor deliverables and measure extraction reliability against the ground truth
• Identify edge cases, structural mismatches, and hallucinations that surface-level review would miss, and document them
• Review existing system interfaces to determine what each screen stores, and build the gold field and schema catalog
• Compile findings into quality-acceptance reports used by the team to sign off on deliveries
• Scope may extend beyond the responsibilities listed above

Requirements

• Demonstrated experience building and shipping a data pipeline (personal, academic, or hackathon) to completion
• Working knowledge of Python for processing structured data (JSON/CSV)
• Familiarity with evaluation concepts (precision, recall, accuracy) and LLM evaluation metrics
• Willingness to learn domain knowledge (e.g. patents)
• Basic familiarity with version control (git)
• Willingness to use AI tools (Claude, Antigravity) daily
• Nice to have: basic SQL; basic cloud experience (GCP/AWS); familiarity with data engineering concepts; comfortable reading and working in English and Chinese