Senior GenAI Quality Engineer and Solution Analyst

re-zoo-me; Singapore; Hybrid; SGD 140,000 - 200,000; Full time

Job summary

re-zoo-me in Singapore is seeking a Senior GenAI Quality Engineer and Solution Analyst to lead testing across UI, API and GenAI components. You will drive end-to-end quality strategies, design tests, and shepherd observable release governance for complex enterprise solutions.

You will validate GenAI outputs, orchestrate agentic test scenarios, and collaborate with product, engineering and security teams to mitigate risk while advancing reliable, user-centered AI applications.

Qualifications

• 5 to 8 years in software quality engineering or similar hands-on role
• Strong experience testing web UIs, backend services and REST APIs
• Hands-on with API tools and automation frameworks (Postman, REST Assured, pytest, Playwright, Cypress, Selenium)
• Proficiency in Java, Python, JavaScript or TypeScript for test automation
• Experience validating distributed systems, including asynchronous processing and queues
• Knowledge of security and privacy testing fundamentals
• Excellent stakeholder communication and design for GenAI projects

Responsibilities

• Define and execute end-to-end test strategies covering UI workflows, backend APIs, integrations and agentic interfaces
• Test conversational and agentic behaviour including multi-turn context, tool interaction, state transitions and recovery from partial failure
• Validate GenAI responses for task completion, grounding, relevance and safe failure
• Perform functional, integration, regression, exploratory, negative, resilience and basic performance testing across app layers
• Design API tests for contracts, authentication, authorization, error handling and rate limits
• Test UI behaviour across browsers and realistic user journeys
• Create and maintain test data, reusable test scenarios and traceable evidence for release governance
• Use logs, traces and observability tools to isolate defects across components
• Automate tests to reduce cycle time and production risk
• Communicate defects and quality risks to engineers, product owners and stakeholders
• Provide evidence-based release recommendations and monitor residual risks
• Collaborate with product owners, engineers and GenAI specialists to define user journeys and outcomes
• Analyse GenAI use cases to determine deterministic vs. agent-based flows
• Translate requirements into end-to-end solution flows and acceptance criteria
• Map interactions across UI, APIs, models, prompts and data sources
• Analyse tradeoffs related to quality, cost, latency and security
• Identify ownership gaps and operational risks before development
• Support human approval and fallback paths for agentic solutions
• Define measurable success criteria for business outcomes and reliability
• Maintain traceability from business need to release evidence
• Facilitate design reviews with process flows and interface specs

Skills

UI & API Quality Engineering

Agentic & GenAI Testing

Risk-based Test Automation & Observab

Tools

Postman

REST Assured

pytest

Playwright

Cypress

Selenium

Git

CI/CD pipelines

Job description

Experience

6-9 Years

Role

Senior GenAI Quality Engineer and Solution Analyst

Key Skills

• UI and API Quality Engineering
• Agentic and GenAI Application Testing
• Risk Based Test Automation and Observability

Responsibilities

• Define and execute end to end test strategiescovering UI workflows, backend APIs, integrations and agentic interfaces.
• Test conversational and agentic behaviour including multi turn context, tool selection, tool inputs and outputs, state transitions, retries, timeouts, handoffs, approvals and recovery from partial failure.
• Validate GenAI responses for task completion, grounding, relevance, consistency, citation behaviour and safe failure, while recognising that outputs can be non deterministic.
• Perform functional, integration, regression, exploratory, negative, resilience and basic performance testing acrossapplication layers.
• Design API tests for contracts, authentication, authorisation, validation, error handling, idempotency, rate limits and downstream failures.
• Test UI behaviour across browsers and realistic user journeys, including loading states, interrupted sessions, feedback capture, accessibility basics and clear error communication.
• Create and maintain test data, reusable test scenarios and traceable evidence suitable for enterprise release governance.
• Use logs, traces, request and response payloads and observability tools to isolate defects and distinguish application, model, data, integration and platform issues.
• Automate the tests that materially reduce cycle time, manual effort or production risk, and keep unstable or low value scenarios out of the automation suite.
• Communicate defects and quality risks clearly to engineers, product owners, GenAI specialists, security teams and business stakeholders.
• Provide an evidence based release recommendation, including known limitations, residual risks and areas requiring monitoring.
• Partner with product owners, business users, architects, engineers and GenAI specialists to define the problem, target user journeys and expected business outcomes.
• Analyse proposed GenAI use cases and determine where deterministic application logic, retrieval, workflow orchestration, tool using agents or human approval should be used.
• Translate business requirements into end to end solution flows, functional requirements, interface behaviors, decision rules, acceptance criteria and non functional requirements.
• Map interactions across user interfaces, APIs, models, prompts, retrieval components, enterprise data sources, agent tools and downstream systems.
• Analyse solution options and document tradeoffs relating to quality, complexity, cost, latency, security, data access, maintainability and operational risk.
• Identify unclear ownership, missing controls, integration assumptions, failure scenarios and operational gaps before development begins.
• Support the design of human approval, fallback, escalation and exception handling paths for agentic solutions.
• Define measurable success criteria covering business outcomes, user experience, functional correctness, response quality, latency, reliability and safe failure.
• Maintain traceability from business need through solution requirement, implementation, evaluation scenario and release evidence.
• Facilitate structured design reviews and communicate findings using process flows, sequence diagrams, interface specifications, decision tables and concise solution documentation.

Requirements

Core Requirements

• 5 to 8 years of experience in software quality engineering, test engineering or a similar hands on role covering complex applications.
• Strong experience testing web user interfaces, backend services and REST APIs.
• Hands on ability with API tools and automation frameworks such as Postman, REST Assured, pytest, Playwright, Cypress, Selenium or equivalent.
• Working knowledge of Java, Python, JavaScriptor TypeScript sufficient to build, review and troubleshoot test automation.
• Strong test analysis skills, including requirements review, risk assessment, boundary analysis, negative testing and traceability.
• Experience validating distributed systems and integrations, including asynchronous processing, queues, batch jobs and downstream dependencies.
• Ability to inspect logs, traces, networkcalls, payloads and database records to identify the actual failure point.
• Experience with Git, pull requests, CI/CDpipelines, test reporting and defect management tools.
• Understanding of security and privacy testing fundamentals, including access control, sensitive data handling, input validation and auditability.
• Strong stakeholder communication and the confidence to challenge weak designs, vague expected outcomes and premature release decisions.
• Ability to work in a fast moving environment where requirements and GenAI behaviour evolve.

Solution analysis and design expectations

• Experience analysing complex applications across user journeys, business processes, APIs, data flows and enterprise integrations.
• Ability to facilitate requirements discussions and convert ambiguous business needs into clear functional requirements, acceptance criteria and solution behaviours.
• Experience producing practical analysis artefacts such as process flows, sequence diagrams, context diagrams, interface specifications, decision tables and user stories.
• Ability to analyse solution alternatives and explain tradeoffs involving quality, cost, performance, security, operational support and delivery complexity.
• Understanding of application architecture concepts including synchronous and asynchronous integrations, event flows, authentication, authorisation, failure handling and system boundaries.
• Ability to distinguish problems that require GenAIfrom those better addressed through deterministic rules, search, workflow automation or conventional application logic.
• Confidence working with product, architecture, engineering, security, data and business stakeholders during discovery and solution design.

GenAI and AgenticTesting Expectations

• Practical understanding of LLM based applications, retrieval augmented generation, prompts, context windows, embeddings and tool using agents.
• Ability to test probabilistic systems using evaluation datasets, repeat runs, quality thresholds and evidence based acceptance criteria rather than brittle exact text matching.
• Experience validating grounded answers, citations, retrieval quality, hallucination risk, prompt injection resistance and safe handling of restricted or unsupported requests.
• Ability to test agent plans and execution paths, tool calls, memory and state, human approval checkpoints, fallback behaviour and termination conditions.
• Awareness of evaluation and observability tooling such as Langfuse, LangSmith, OpenTelemetry, Elastic, Splunk or equivalent.

Nice to Have

• Experience testing applications in banking, finance or another regulated enterprise environment.
• Experience with contract testing, service virtualisation, synthetic monitoring or performance testing tools.
• Experience with Kubernetes, OpenShift, AWShosted services or containerised deployments.
• Accessibility testing experience and familiarity with WCAG based checks.
• Experience building lean quality dashboards that show release risk, defect escape patterns, flaky tests and cycle time.
• Exposure to red teaming or adversarial testing of GenAI applications.