AI QA Lead: LLM, RAG & Safety Testing

NCS Pte Ltd; Hong Kong; On-site; HKD 420,000 - 640,000; Full time

Job summary

NCS Pte Ltd in Hong Kong () is seeking an AI Tester to lead end-to-end evaluation of AI applications, including LLM features, RAG systems, and multi-agent workflows. You will own test design, execution, and reporting across offline, pre-prod, and in-prod stages, collaborating with product, data, and platform teams.

The ideal candidate has 3+ years in software testing/QA with API and performance testing, Python/TypeScript skills, and familiarity with CI/CD. A CS/IT degree is preferred.

Qualifications

• 3+ years in software testing/QA with strong test methodology and tooling.
• Hands-on API testing and performance testing.
• Familiarity with CI/CD and version control.
• Programming familiarity (Python/TypeScript).
• Understanding of ML concepts and ML Ops is a plus.

Responsibilities

• Define evaluation strategies (golden sets, adversarial suites, regressions) and SLOs for quality, safety, latency, and cost.
• Establish rubric-based human reviews and calibrate annotators.
• Instrument LLM-as-judge where appropriate with calibration and spot checks.
• Test retrieval, grounding, and tool-use scenarios across AI features.
• Run token-aware load/soak tests and monitor performance metrics.
• Ensure security, privacy, and guardrails pre/post inference.
• Integrate testing into CI/CD and produce risk-focused reports.

Skills

Software testing

API testing

Performance testing

CI/CD

Python/TypeScript

Education

CS/IT degree or equivalent

Tools

Postman

REST Assured

Playwright

Cypress

Selenium

k6

JMeter

Job description

NCS Pte Ltd in Hong Kong () is seeking an AI Tester to lead end-to-end evaluation of AI applications, including LLM features, RAG systems, and multi-agent workflows. You will own test design, execution, and reporting across offline, pre-prod, and in-prod stages, collaborating with product, data, and platform teams.

The ideal candidate has 3+ years in software testing/QA with API and performance testing, Python/TypeScript skills, and familiarity with CI/CD. A CS/IT degree is preferred.