AI Automation Engineer - Urgent

Private Advertiser; Taguig; Hybrid; PHP 700,000 - 900,000; Full time

Job summary

NCS, a leading AI Tech Services company, is seeking an AI Tester / AI Automation Engineer to evaluate AI models and validate workflows. The role covers traditional QA as well as ML ops, ensuring accuracy, safety, and reliability across production systems.

Responsibilities include building evaluation pipelines, developing automated tests with Python and Pytest, and monitoring models in production for drift. Collaboration with data scientists and developers will refine prompt design and system

Qualifications

• 3+ years in QA or test automation
• 1–2 years directly focused on AI/ML-enabled systems
• Strong scripting skills, with emphasis on Python

Responsibilities

• AI Evaluation: build evaluation pipelines for model behavior, factuality, toxicity, and task success.
• Test Automation: develop and maintain automated test scripts using Python and Pytest.
• System Monitoring: monitor AI models in production for quality degradation and drift.
• Bug Identification & Red Teaming: identify failures, adversarial testing, jailbreaks, and prompt injections.
• Cross-Functional Collaboration: work with data scientists, product managers, and developers to improve prompt design and workflow structure.

Skills

Playwright

AI Testing

Python

Karate Testing

Tools

Selenium

Job description

Company Overview

NCS is a leading AI Tech Services company. With a 15,000-strong team across the Asia Pacific, NCS scales its platforms and capabilities to provide clients with greater agility and AI expertise across a range of industries. Embracing a strong ecosystem of global partners, NCS transforms technology services delivery combining AI with digital resilience to drive real business impact. NCS is a subsidiary of the Singtel Group.

An AI Tester / AI Automation Engineer evaluates and validates AI models and workflows to ensure they are accurate, safe, and reliable. They design tests for both traditional software constraints and non-deterministic AI outputs, checking for hallucinations, bias, and performance. This role bridges standard quality assurance and machine learning operations.

Key Responsibilities

• AI Evaluation: Build evaluation pipelines to assess model behavior, factuality, toxicity, and task success.
• Test Automation: Develop and maintain automated test scripts using Python and frameworks like Pytest.
• System Monitoring: Continuously monitor AI models in production environments for quality degradation and system drift.
• Bug Identification & Red Teaming: Find creative ways to break AI products, including adversarial stress testing, jailbreaks, and prompt injections.
• Cross-Functional Collaboration: Work closely with data scientists, product managers, and developers to improve prompt design and workflow structure.

Required Qualifications & Skills

• w/ experience either of the ff:
  • playwright
  • AI Testing
  • Python Based
  • Karate Testing
• Experience: Typically 3+ years in QA or test automation, with at least 1-2 years directly focused on AI/ML-enabled systems.
• Programming: Strong scripting skills, heavily preferring Python.
• Automation Tools: Proficiency in UI and workflow test automation using tools like Playwright or Selenium.
• AI Tooling: Familiarity with LLM evaluation and observability frameworks like DeepEval, Ragas, or Evidently.AI.
• Analytical Mindset: Excellent debugging skills with an eye for spotting data anomalies and edge-case failures