AI QA Engineer: AI Testing & Quality Frameworks

State Street

Burlington (MA)

On-site

USD 80,000 - 140,000

Full time

8 days ago

Application generator

Get a reply from this employer — a resume and cover letter tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

401K with company match

Medical, dental, vision

Paid time off

Job summary

State Street's Charles River Development/State Street Alpha is seeking a Quality Assurance Engineer focused on AI testing to design scalable QA frameworks and validate AI-driven product behavior within CRIMS/Alpha. You will work in a scrum team with product, software engineers, and architects to ensure AI-enabled capabilities are testable, reliable, and aligned with client needs.

The role emphasizes testing AI guardrails, model-driven outputs, and integration across CI/CD pipelines, with

Qualifications

Hands-on AI testing experience within software products.
Experience validating AI-driven features and outputs.
Familiarity with QA methodologies, test strategies, and automation best practices.
Ability to translate ambiguous behavior into test scenarios and acceptance criteria.
Experience integrating tests into CI/CD pipelines.

Responsibilities

Develop and maintain automated and manual test frameworks for AI-enabled features.
Create test plans, cases, and data strategies for functional and non-functional testing.
Validate AI outputs, workflow integration, and user experience impacts.
Design tests for AI guardrails, probabilistic outputs, and boundaries.
Collaborate to define acceptance criteria and quality gates for AI stories.
Support production monitoring, drift detection, and feedback loops.
Drive defect analysis, root-cause investigation, and timely resolutions.
Integrate automated testing into the team's CI/CD pipelines.