AI QA & Automation Engineer: Test, Break & Improve Models

Private Advertiser; Taguig; Hybrid; PHP 700,000 - 900,000; Full time

Job summary

NCS, a leading AI Tech Services company, is seeking an AI Tester / AI Automation Engineer to evaluate AI models and validate workflows. The role covers traditional QA as well as ML ops, ensuring accuracy, safety, and reliability across production systems.

Responsibilities include building evaluation pipelines, developing automated tests with Python and Pytest, and monitoring models in production for drift. Collaboration with data scientists and developers will refine prompt design and system

Qualifications

• 3+ years in QA or test automation
• 1–2 years directly focused on AI/ML-enabled systems
• Strong scripting skills, with emphasis on Python

Responsibilities

• AI Evaluation: build evaluation pipelines for model behavior, factuality, toxicity, and task success.
• Test Automation: develop and maintain automated test scripts using Python and Pytest.
• System Monitoring: monitor AI models in production for quality degradation and drift.
• Bug Identification & Red Teaming: identify failures, adversarial testing, jailbreaks, and prompt injections.
• Cross-Functional Collaboration: work with data scientists, product managers, and developers to improve prompt design and workflow structure.

Skills

Playwright

AI Testing

Python

Karate Testing

Tools

Selenium

Job description

NCS, a leading AI Tech Services company, is seeking an AI Tester / AI Automation Engineer to evaluate AI models and validate workflows. The role covers traditional QA as well as ML ops, ensuring accuracy, safety, and reliability across production systems.

Responsibilities include building evaluation pipelines, developing automated tests with Python and Pytest, and monitoring models in production for drift. Collaboration with data scientists and developers will refine prompt design and system