Remote NLP QA Engineer for AI Conversational Testing

BruntWork; Philippines; Remote; PHP 600,000 - 1,200,000; Full time

Benefits offered by this job

Permanent work from home

Immediate hiring

Health Insurance Coverage for eligible

Job summary

BruntWork is seeking a detailed NLP QA Engineer to ensure reliability, accuracy, and safety of Candid Chat AI across web and mobile platforms. You will design, execute, and automate tests for LLMs, conversational flows, and system integrations.

You will work within CI/CD pipelines, perform adversarial and safety testing, and document AI-specific issues with clear reproduction steps for ML and engineering teams. 3+ years in QA for conversational AI and strong API/UI automation skills are required.

Qualifications

• 3+ years of Software Quality Assurance experience, focusing on Conversational AI or LLM-powered apps.
• Proven track record in testing NLP models, prompt safety, and hallucination evaluation.

Responsibilities

• Test end-to-end conversational AI flows across web and mobile interfaces.
• Evaluate AI responses for factual accuracy, context relevance, and tone consistency.
• Perform red-teaming and adversarial testing, including prompt injection and guardrail verification.
• Design and execute regression test suites when models or backends are updated.
• Build and maintain UI and API automated testing frameworks.
• Create benchmark datasets to track AI output quality over time.
• Integrate automated AI quality and regression checks into CI/CD pipelines.
• Document AI-specific bugs with reproduction steps for ML and Engineering teams.

Skills

NLP QA

API testing

UI automation

Adversarial testing

Documentation & analysis

Communication

Tools

Postman

REST Assured

Python

Selenium

Playwright

Appium

Cypress

Job description

BruntWork is seeking a detailed NLP QA Engineer to ensure reliability, accuracy, and safety of Candid Chat AI across web and mobile platforms. You will design, execute, and automate tests for LLMs, conversational flows, and system integrations.

You will work within CI/CD pipelines, perform adversarial and safety testing, and document AI-specific issues with clear reproduction steps for ML and engineering teams. 3+ years in QA for conversational AI and strong API/UI automation skills are required.