Applied AI Engineer, Evaluations Intern

NUS Enterprise; Singapore; In loco; SGD 60.000 - 100.000; Tempo pieno

Descrizione del lavoro

NUSX Labs is the technical arm of NUSX, turning research into inventions, patents, and licensed deals. We are a hands-on team where every AI system shipped is graded against a ground-truth dataset.

You will build, evaluate, and improve the data pipeline and benchmarking framework under direct supervision of an engineer. Join a small team with real stakeholder-facing projects and deployment experience, applying Python, Claude, and Antigravity to develop high-quality data tools for patent-related

Competenze

• Proven ability to build and ship a data pipeline.
• Familiarity with evaluation concepts (precision, recall, accuracy) and LLM evaluation metrics.
• Willingness to learn domain knowledge (e.g., patents).
• Basic familiarity with version control (Git).
• Willingness to use AI tools (Claude, Antigravity) daily.
• Nice to have: basic SQL; cloud familiarity (GCP/AWS); understanding data engineering concepts; comfortable reading/writing in English and Chinese.

Mansioni

• Build benchmark datasets across multiple document types using Claude and Antigravity, drafting and reviewing with an annotation tool.
• Run the blind-test evaluation matrix on vendor deliverables and compare against ground truth.
• Identify edge cases, structural mismatches, and hallucinations and document them.
• Review system interfaces to determine data storage for screens and build gold field and schema catalogs.
• Compile findings into quality-acceptance reports for team sign-off on deliveries.
• Scope may extend beyond listed responsibilities.

Strumenti

Python

Git

Claude

Antigravity

SQL

Descrizione del lavoro

NUSX Labs is NUSX's technical arm, turning NUS research into inventions, patents, and licensed deals. We're a small, hands-on team, and every AI system we ship is graded against a ground-truth dataset built by hand: for six months, you're the one building it, and your judgments become the benchmark the next model is measured against. It's judgment work, not data entry, across an LLM pipeline that reads patent documents into structured data, a natural-language query layer, and early-signal deal-sourcing tools. You'll be supervised directly by an engineer on the team, with real stakeholder-facing projects and deployment experience.

Roles and Responsibilities

• Build benchmark datasets used to evaluate model performance, across approximately 23 document types, using Claude and Antigravity to draft and an annotation tool to review and refine.
• Run the blind-test evaluation matrix on vendor deliverables and measure extraction reliability against the ground truth.
• Identify edge cases, structural mismatches, and hallucinations that surface-level review would miss, and document them.
• Review existing system interfaces to determine what each screen stores, and build the gold field and schema catalog.
• Compile findings into quality-acceptance reports used by the team to sign off on deliveries.
• Scope may extend beyond the responsibilities listed above.

• Demonstrated experience building and shipping a data pipeline (personal, academic, or hackathon) to completion.
• Working knowledge of Python for processing structured data (JSON/CSV). Familiarity with evaluation concepts (precision, recall, accuracy) and LLM evaluation metrics.
• Willingness to learn domain knowledge (e.g. patents).
• Basic familiarity with version control (git).
• Willingness to use AI tools (Claude, Antigravity) daily.
• Nice to have: basic SQL; basic cloud experience (GCP/AWS); familiarity with data engineering concepts; comfortable reading and working in English and Chinese.