Senior GenAI Quality Engineer and Solution Analyst

SAKSOFT LIMITED; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

SAKSOFT LIMITED in Singapore is seeking a Senior GenAI Quality Engineer and Solution Analyst to lead end-to-end testing across UI, APIs and agentic GenAI interfaces.

You will design test strategies, validate probabilistic GenAI outputs, and collaborate with engineers, product owners and GenAI specialists to ensure robust release readiness in a fast-evolving environment.

Qualifications

• 5 to 8 years of experience in software quality engineering, test engineering or a similar hands on role covering complex applications.
• Strong experience testing web user interfaces, backend services and REST APIs.
• Hands on ability with API tools and automation frameworks such as Postman, REST Assured, pytest, Playwright, Cypress, Selenium.
• Working knowledge of Java, Python, JavaScript or TypeScript sufficient to build, review and troubleshoot test automation.
• Strong test analysis skills, including requirements review, risk assessment, boundary analysis, negative testing and traceability.
• Experience validating distributed systems and integrations, including asynchronous processing, queues, batch jobs and downstream dependencies.
• Ability to inspect logs, traces, network calls, payloads and database records to identify the actual failure point.
• Experience with Git, pull requests, CI/CD pipelines, test reporting and defect management tools.
• Understanding of security and privacy testing fundamentals, including access control, sensitive data handling, input validation and auditability.
• Strong stakeholder communication and the confidence to challenge weak designs, vague expected outcomes and premature release decisions.
• Ability to work in a fast moving environment where requirements and GenAI behaviour evolve.

Responsibilities

• Define and execute end to end test strategies covering UI workflows, backend APIs, integrations and agentic interfaces.
• Test conversational and agentic behaviour including multi turn context, tool selection, tool inputs and outputs, state transitions, retries, timeouts, handoffs, approvals and recovery from partial failure.
• Validate GenAI responses for task completion, grounding, relevance, citation behaviour and safe failure, while recognising that outputs can be non deterministic.
• Perform functional, integration, regression, exploratory, negative, resilience and basic performance testing across application layers.
• Design API tests for contracts, authentication, authorisation, validation, error handling, idempotency, rate limits and downstream failures.
• Test UI behaviour across browsers and realistic user journeys, including loading states, interrupted sessions, feedback capture, accessibility basics and clear error communication.
• Create and maintain test data, reusable test scenarios and traceable evidence suitable for enterprise release governance.
• Use logs, traces, request and response payloads and observability tools to isolate defects and distinguish application, model, data, integration and platform issues.
• Automate the tests that materially reduce cycle time, manual effort or production risk, and keep unstable or low value scenarios out of the automation suite.
• Communicate defects and quality risks clearly to engineers, product owners, GenAI specialists, security teams and business stakeholders.
• Provide an evidence based release recommendation, including known limitations, residual risks and areas requiring monitoring.
• Partner with product owners, business users, architects, engineers and GenAI specialists to define the problem, target user journeys and expected business outcomes.
• Analyse proposed GenAI use cases and determine where deterministic application logic, retrieval, workflow orchestration, tool using agents or human approval should be used.
• Translate business requirements into end to end solution flows, functional requirements, interface behaviours, decision rules, acceptance criteria and non functional requirements.
• Map interactions across user interfaces, APIs, models, prompts, retrieval components, enterprise data sources, agent tools and downstream systems.
• Analyse solution options and document tradeoffs relating to quality, complexity, cost, latency, security, data access, maintainability and operational risk.
• Identify unclear ownership, missing controls, integration assumptions, failure scenarios and operational gaps before development begins.
• Support the design of human approval, fallback, escalation and exception handling paths for agentic solutions.
• Define measurable success criteria covering business outcomes, user experience, functional correctness, response quality, latency, reliability and safe failure.
• Maintain traceability from business need through solution requirement, implementation, evaluation scenario and release evidence.
• Facilitate structured design reviews and communicate findings using process flows, sequence diagrams, interface specifications, decision tables and concise solution documentation.

Skills

UI testing

API testing

Automation frameworks

GenAI testing

Observability

Risk based testing

Exploratory testing

Tools

Postman

REST Assured

pytest

Playwright

Cypress

Selenium

Job description

Experience: 6-9 Years

Role: Senior GenAI Quality Engineer and Solution Analyst

Key Skills

• UI and API Quality Engineering
• Agentic and GenAI Application Testing
• Risk Based Test Automation and Observability

Responsibilities

• Define and execute end to end test strategiescovering UI workflows, backend APIs, integrations and agentic interfaces.
• Test conversational and agentic behaviour including multi turn context, tool selection, tool inputs and outputs, state transitions, retries, timeouts, handoffs, approvals and recovery from partial failure.
• Validate GenAI responses for task completion, grounding, relevance, consistency, citation behaviour and safe failure, while recognising that outputs can be non deterministic.
• Perform functional, integration, regression, exploratory, negative, resilience and basic performance testing acrossapplication layers.
• Design API tests for contracts, authentication, authorisation, validation, error handling, idempotency, rate limits and downstream failures.
• Test UI behaviour across browsers and realistic user journeys, including loading states, interrupted sessions, feedback capture, accessibility basics and clear error communication.
• Create and maintain test data, reusable test scenarios and traceable evidence suitable for enterprise release governance.
• Use logs, traces, request and response payloads and observability tools to isolate defects and distinguish application, model, data, integration and platform issues.
• Automate the tests that materially reduce cycle time, manual effort or production risk, and keep unstable or low value scenarios out of the automation suite.
• Communicate defects and quality risks clearly to engineers, product owners, GenAI specialists, security teams and business stakeholders.
• Provide an evidence based release recommendation, including known limitations, residual risks and areas requiring monitoring.
• Partner with product owners, business users, architects, engineers and GenAI specialists to define the problem, target user journeys and expected business outcomes.
• Analyse proposed GenAI use cases and determine where deterministic application logic, retrieval, workflow orchestration, tool using agents or human approval should be used.
• Translate business requirements into end to end solution flows, functional requirements, interface behaviours, decision rules, acceptance criteria and non functional requirements.
• Map interactions across user interfaces, APIs, models, prompts, retrieval components, enterprise data sources, agent tools and downstream systems.
• Analyse solution options and document tradeoffs relating to quality, complexity, cost, latency, security, data access, maintainability and operational risk.
• Identify unclear ownership, missing controls, integration assumptions, failure scenarios and operational gaps before development begins.
• Support the design of human approval, fallback, escalation and exception handling paths for agentic solutions.
• Define measurable success criteria covering business outcomes, user experience, functional correctness, response quality, latency, reliability and safe failure.
• Maintain traceability from business need through solution requirement, implementation, evaluation scenario and release evidence.
• Facilitate structured design reviews and communicate findings using process flows, sequence diagrams, interface specifications, decision tables and concise solution documentation.

Requirements

Core Requirements

• 5 to 8 years of experience in software quality engineering, test engineering or a similar hands on role covering complex applications.
• Strong experience testing web user interfaces, backend services and REST APIs.
• Hands on ability with API tools and automation frameworks such as Postman, REST Assured, pytest, Playwright, Cypress, Selenium or equivalent.
• Working knowledge of Java, Python, JavaScriptor TypeScript sufficient to build, review and troubleshoot test automation.
• Strong test analysis skills, including requirements review, risk assessment, boundary analysis, negative testing and traceability.
• Experience validating distributed systems and integrations, including asynchronous processing, queues, batch jobs and downstream dependencies.
• Ability to inspect logs, traces, networkcalls, payloads and database records to identify the actual failure point.
• Experience with Git, pull requests, CI/CDpipelines, test reporting and defect management tools.
• Understanding of security and privacy testing fundamentals, including access control, sensitive data handling, input validation and auditability.
• Strong stakeholder communication and the confidence to challenge weak designs, vague expected outcomes and premature release decisions.
• Ability to work in a fast moving environment where requirements and GenAI behaviour evolve.

Solution analysis and design expectations

• Experience analysing complex applications across user journeys, business processes, APIs, data flows and enterprise integrations.
• Ability to facilitate requirements discussions and convert ambiguous business needs into clear functional requirements, acceptance criteria and solution behaviours.
• Experience producing practical analysis artefacts such as process flows, sequence diagrams, context diagrams, interface specifications, decision tables and user stories.
• Ability to analyse solution alternatives and explain tradeoffs involving quality, cost, performance, security, operational support and delivery complexity.
• Understanding of application architecture concepts including synchronous and asynchronous integrations, event flows, authentication, authorisation, failure handling and system boundaries.
• Ability to distinguish problems that require GenAIfrom those better addressed through deterministic rules, search, workflow automation or conventional application logic.
• Confidence working with product, architecture, engineering, security, data and business stakeholders during discovery and solution design.

GenAI and AgenticTesting Expectations

• Practical understanding of LLM based applications, retrieval augmented generation, prompts, context windows, embeddings and tool using agents.
• Ability to test probabilistic systems using evaluation datasets, repeat runs, quality thresholds and evidence based acceptance criteria rather than brittle exact text matching.
• Experience validating grounded answers, citations, retrieval quality, hallucination risk, prompt injection resistance and safe handling of restricted or unsupported requests.
• Ability to test agent plans and execution paths, tool calls, memory and state, human approval checkpoints, fallback behaviour and termination conditions.
• Awareness of evaluation and observability tooling such as Langfuse, LangSmith, OpenTelemetry, Elastic, Splunk or equivalent.

Nice to Have

• Experience testing applications in banking, finance or another regulated enterprise environment.
• Experience with contract testing, service virtualisation, synthetic monitoring or performance testing tools.
• Experience with Kubernetes, OpenShift, AWShosted services or containerised deployments.
• Accessibility testing experience and familiarity with WCAG based checks.
• Experience building lean quality dashboards that show release risk, defect escape patterns, flaky tests and cycle time.
• Exposure to red teaming or adversarial testing of GenAI applications.