AI QA Engineer: Enterprise AI Testing & Frameworks

STATE STREET CORPORATION; United States; On-site; USD 80,000 - 140,000; Full time

Benefits offered by this job

401K with company match

Insurance: medical, dental, vision

Life insurance

Paid time off

Employee Assistance Program

Bonus / incentive opportunities

Job summary

State Street Corporation is seeking a Quality Assurance Engineer – AI Testing within Charles River Development/State Street Alpha to strengthen AI-driven testing across enterprise products. You will join a scrum team to design scalable QA frameworks, validate AI behavior, and drive rigorous quality controls for AI-enabled features.

The ideal candidate has up to 2 years in QA, hands-on AI testing experience, strong testing fundamentals, and a proactive approach to integrating AI into test design

Qualifications

• 0-2 years of professional software testing or quality assurance experience, preferably within agile or scrum-based product development teams.
• Proven experience implementing AI capabilities within the software testing process, such as intelligent test generation, test optimization, defect analysis, or productivity acceleration.
• Hands-on experience testing AI functionality embedded within a software product, including validation of generated outputs, model-driven behavior, workflow integration, and exception scenarios.
• Understanding of AI-specific testing concepts beyond deterministic validation, including guardrail testing, probabilistic output assessment, accuracy evaluation, and monitoring for production drift or model behavior changes over time.
• Strong QA methodologies, test strategy development, defect lifecycle management, and test automation best practices.
• Experience building or extending test frameworks to support scalable, maintainable, and reusable testing practices.
• Experience with API testing, integration testing, regression testing, and end-to-end validation in complex enterprise applications.
• Working knowledge of test automation tools, scripting, and CI/CD integration.

Responsibilities

• Develop and maintain automated and manual test frameworks that support validation of AI-enabled features across the CRIMS / Alpha platform.
• Create test plans, test cases, and test data strategies for functional, integration, regression, and non-functional testing of AI-driven capabilities.
• Validate product behavior for AI use cases, including quality of outputs, workflow integration, response accuracy, exception handling, and user experience impacts.
• Design and execute tests for AI guardrails, probabilistic outputs, and response boundaries to ensure generated results are accurate, safe, consistent, and aligned with intended business outcomes.
• Help define approaches for monitoring AI behavior in production, including drift detection, output quality trends, and feedback loops to confirm AI-enabled features continue learning and performing as expected over time.
• Identify, document, and drive resolution of defects, with clear analysis of reproducibility, business impact, and root cause patterns.
• Work with engineering teams to integrate automated testing into CI/CD pipelines and improve test coverage across the scrum team’s delivery scope.
• Apply AI tools and techniques within the test lifecycle to improve productivity, accelerate coverage creation, and enhance defect discovery.
• Collaborate with developers and product partners to define measurable acceptance criteria and quality gates for AI-related user stories and features.
• Support risk-based testing and help prioritize test efforts across new development, regression areas, and high-impact client workflows.
• Contribute to continuous improvement of QA standards, test assets, documentation, and team-level best practices for testing AI-enabled products.
• Communicate testing progress, risks, and quality insights effectively within the scrum team and to broader stakeholders as needed.

Skills

AI testing

Test automation

API testing

CI/CD

Analytical skills

Communication

Education

Bachelor’s degree in Computer Science, Engineering, Information Systems, or related field

Tools

Scripting

CI/CD tooling

API testing tools

Job description

State Street Corporation is seeking a Quality Assurance Engineer – AI Testing within Charles River Development/State Street Alpha to strengthen AI-driven testing across enterprise products. You will join a scrum team to design scalable QA frameworks, validate AI behavior, and drive rigorous quality controls for AI-enabled features.

The ideal candidate has up to 2 years in QA, hands-on AI testing experience, strong testing fundamentals, and a proactive approach to integrating AI into test design