AI-Driven QA Engineer: Test AI-Powered Risk Tools

Moody's Corporation; Charlotte (NC); On-site; USD 78,000 - 114,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

Parental leave

PTO

401(k) plan

Life insurance

Disability insurance

Stock purchase plan

Tuition reimbursement

Job summary

Moody's Corporation is seeking a quality engineering professional to support software and AI product testing within an Agile team. You will document findings, contribute to QA process improvements, and help ensure AI outputs are accurate and reliable.

The role emphasizes manual, exploratory, and API testing of AI-enabled features, with a focus on documenting defects clearly and collaborating with engineers and product owners.

Qualifications

• Basics of software testing concepts including manual, exploratory, regression, API, and integration testing methodologies.
• Basic programming or scripting knowledge, preferably in Python, with familiarity in APIs, databases, JSON, version control, and defect tracking tools.
• Basic understanding of AI concepts, large language models, and common AI limitations such as hallucinations, incomplete responses, and inconsistent outputs.
• Interest in prompt engineering, context engineering, and the evaluation of AI-enabled products.
• Ability to document defects clearly with steps to reproduce, expected results, and actual results.
• Strong analytical, communication, documentation, and problem-solving skills with the ability to collaborate across engineering and product teams.
• Basic understanding of artificial intelligence concepts, with curiosity and enthusiasm for learning how AI tools can be used to improve processes and drive efficiency. Interest in exploring AI systems and a willingness to develop awareness of responsible AI practices, including risk management and ethical use

Responsibilities

• Support quality engineering efforts across an Agile team by executing software and AI product testing, documenting findings, and contributing to continuous improvement of QA processes.
• Perform manual, exploratory, regression, API, and integration testing within an Agile team environment under close guidance.
• Test software features that include AI or large language model components, reviewing outputs for accuracy, relevance, completeness, consistency, and instruction-following.
• Create basic test scenarios, prompts, and test data for evaluating AI behavior and identifying issues such as hallucinations, incomplete responses, and inconsistent results.
• Apply basic prompt engineering and context engineering techniques during testing activities.
• Use AI-assisted tools such as Codex, Claude Code, and similar AI command-line interface tools under the guidance of senior team members.
• Assist with existing AI evaluation harnesses, test scripts, and regression checks to support quality validation efforts.
• Document defects with clear steps to reproduce, expected results, and actual results, and collaborate with engineers, developers, and product owners to investigate and resolve issues.
• Contribute to test documentation, reusable test cases, and the continuous improvement of quality assurance processes and standards

Skills

Software testing concepts

Python programming

APIs

JSON

Version control

Defect tracking tools

AI concepts

Prompt engineering

Responsible AI

Communication & collaboration

Education

Bachelor's degree in Computer Science, Software Engineering, or a related field

Tools

Codex

Claude Code

Git

Job description

Moody's Corporation is seeking a quality engineering professional to support software and AI product testing within an Agile team. You will document findings, contribute to QA process improvements, and help ensure AI outputs are accurate and reliable.

The role emphasizes manual, exploratory, and API testing of AI-enabled features, with a focus on documenting defects clearly and collaborating with engineers and product owners.