AI Quality Assurance Architect (Europe)

CHI Software; United States; Remote; USD 150,000 - 190,000; Full time

Benefits offered by this job

Remote work options

Medical insurance for Ukraine staff

Legal support

Free English classes

Friendly atmosphere

Vacation allowance

Job summary

CHI Software, a global software development services provider, seeks an AI Quality Assurance Architect in the United States to lead end-to-end test and quality architecture across APIs, integrations, and AI/LLM systems. You will define strategies, governance, and automation, coordinating across multiple teams and vendors.

You will establish quality gates, promote traceability from requirements to tests, and drive early quality practices in architecture and design, ensuring robust, scalable

Qualifications

• 10+ years of experience in software testing or test architecture.
• Proven experience as Test Architect or Senior QA Lead.
• Experience defining test strategies across complex enterprise systems and multiple teams.
• Strong knowledge of Agile, DevOps/DevSecOps, CI/CD, and automated quality gates.
• Hands-on with BDD, Gherkin, Given/When/Then scenarios, and requirements traceability.
• Experience integrating automated BDD tests into CI/CD pipelines.
• Experience testing AI/LLM or agentic systems, including evaluation frameworks, regression testing, hallucination risks, guardrails, and safety.
• Strong knowledge of API, integration, contract, performance, and resilience testing.
• Strong stakeholder management and technical leadership skills.

Responsibilities

• Own and evolve the end-to-end test and quality architecture.
• Define test strategies covering APIs, integration, automation, regression, performance, and production assurance.
• Establish automation frameworks, quality gates, and release criteria.
• Implement BDD standards and ensure traceability from requirements to automated tests.
• Integrate quality practices early into requirements, architecture, and design.
• Define AI/LLM testing approaches, evaluation datasets, regression suites, and safety checks.
• Ensure coverage of APIs, data flows, third-party integrations, and distributed systems.
• Define test-data and environment strategies.
• Establish quality metrics, identify systemic risks, and coach QA teams

Skills

Test architecture

Quality engineering

Agile

DevOps/DevSecOps

CI/CD

BDD

Gherkin

Traceability

Automation testing

AI/LLM testing

API testing

Integration testing

Performance testing

Resilience testing

Stakeholder management

Technical leadership

Tools

AWS

Azure

GCP

Job description

CHI Software is a global software development services provider delivering AI-driven solutions, cloud services, data engineering across EdTech, FinTech and HealthTech. We help businesses modernize, scale and unlock new value through the latest technologies.
We are looking for AI Quality Assurance Architect

Requirements

• 10+ years of experience in software testing, quality engineering, or test architecture.
• Proven experience as a Test Architect, Quality Architect, Principal QA Engineer, or Senior Test Lead.
• Experience defining test strategies across complex enterprise systems and multiple teams.
• Strong knowledge of Agile, DevOps/DevSecOps, CI/CD, and automated quality gates.
• Hands-on experience with BDD, Gherkin, Given/When/Then scenarios, and requirements traceability.
• Experience integrating automated BDD tests into CI/CD pipelines.
• Experience testing AI/LLM or agentic systems, including evaluation frameworks, regression testing, hallucination risks, guardrails, and safety.
• Strong knowledge of API, integration, contract, performance, and resilience testing.
• Strong stakeholder management and technical leadership skills.

Responsibilities

• Own and evolve the end-to-end test and quality architecture.
• Define test strategies covering APIs, integration, automation, regression, performance, and production assurance.
• Establish automation frameworks, quality gates, and release criteria.
• Implement BDD standards and ensure traceability from requirements to automated tests.
• Integrate quality practices early into requirements, architecture, and design.
• Define AI/LLM testing approaches, evaluation datasets, regression suites, and safety checks.
• Ensure coverage of APIs, data flows, third-party integrations, and distributed systems.
• Define test-data and environment strategies.
• Establish quality metrics, identify systemic risks, and coach QA teams

Will be a plus

• Knowledge of microservices, event-driven architectures, and API-first platforms.
• Experience with AWS, Azure, or GCP.
• Experience with data-quality testing.
• Experience with AI/ML, Generative AI, RAG, or agentic systems.
• Experience establishing quality standards across multiple teams or suppliers.
• Experience transitioning from manual QA to automated quality engineering.
• Covered vacation period: 20 business days, 5 sick leave days, and 3 days off
• Free English classes
• Truly friendly and supporting atmosphere
• Working remotely or in one of our offices
• Medical insurance for employees from Ukraine
• Legal support