AI QA Engineer

PowerToFly; Charlotte (NC); On-site; USD 78,000 - 114,000; Full time

Benefits offered by this job

Medical

Dental

Vision

Parental leave

Paid time off

401(k) plan with employee and company"

Life insurance

Disability insurance

Discounted employee stock purchasepl

Tuition reimbursement

Job summary

Moody's is seeking a Quality Engineer to support Agile QA for software and AI-enabled products, focusing on manual, exploratory, regression, API, and integration testing within a collaborative team.

You will document defects, create test data and prompts for evaluating AI behavior, and contribute to improving QA processes while partnering with engineering, data science, and product teams to deliver reliable risk assessment tools.

Qualifications

• Basic understanding of software testing concepts across manual, exploratory, regression, API, and integration testing.
• Familiarity with APIs, databases, JSON, version control, and defect tracking tools.
• Basic understanding of AI concepts, large language models, and responsible AI practices.

Responsibilities

• Perform manual, exploratory, regression, API, and integration testing within an Agile team.
• Test features with AI/LLM components for accuracy, relevance, completeness, consistency, and instruction-following.
• Create test scenarios, prompts, and data for evaluating AI behavior and identifying issues such as hallucinations, incomplete responses, and inconsistent results.
• Apply basic prompt engineering and context engineering techniques during testing activities.
• Use AI-assisted tools under the guidance of senior team members.
• Assist with existing AI evaluation harnesses, test scripts, and regression checks to support quality validation efforts.
• Document defects with clear steps to reproduce, expected results, and actual results, and collaborate with engineers, developers, and product owners to investigate and resolve issues.
• Contribute to test documentation, reusable test cases, and the continuous improvement of quality assurance processes and standards.

Skills

Software testing concepts

Python basics

APIs & JSON

AI concepts & LLMs

Prompt engineering

Defect documentation

Collaborative problem solving

Education

Bachelor's degree in CS/SE or related

Tools

Codex

Claude Code

AI tooling CLI

Job description

At Moody's, we unite the brightest minds to turn today’s risks into tomorrow’s opportunities. We do this by striving to create an inclusive environment where everyone feels welcome to be who they are—with the freedom to exchange ideas, think innovatively, and listen to each other and customers in meaningful ways. Moody’s is transforming how the world sees risk. As a global leader in ratings and integrated risk assessment, we’re advancing AI to move from insight to action—enabling intelligence that not only understands complexity but responds to it. We decode risk to unlock opportunity, helping our clients navigate uncertainty with clarity, speed, and confidence.

Skills and Competencies

• Basic understanding of software testing concepts, including manual, exploratory, regression, API, and integration testing methodologies
• Basic programming or scripting knowledge, preferably in Python, with familiarity in APIs, databases, JSON, version control, and defect tracking tools
• Basic understanding of AI concepts, large language models, and common AI limitations such as hallucinations, incomplete responses, and inconsistent outputs
• Interest in prompt engineering, context engineering, and the evaluation of AI-enabled products
• Ability to document defects clearly with steps to reproduce, expected results, and actual results
• Strong analytical, communication, documentation, and problem-solving skills with the ability to collaborate across engineering and product teams
• Basic understanding of artificial intelligence concepts, with curiosity and enthusiasm for learning how AI tools can be used to improve processes and drive efficiency. Interest in exploring AI systems and a willingness to develop awareness of responsible AI practices, including risk management and ethical use

Education

• Bachelor's degree in Computer Science, Software Engineering, or a related field

Responsibilities

Support quality engineering efforts across an Agile team by executing software and AI product testing, documenting findings, and contributing to continuous improvement of QA processes.

• Perform manual, exploratory, regression, API, and integration testing within an Agile team environment under close guidance
• Test software features that include AI or large language model components, reviewing outputs for accuracy, relevance, completeness, consistency, and instruction-following
• Create basic test scenarios, prompts, and test data for evaluating AI behavior and identifying issues such as hallucinations, incomplete responses, and inconsistent results
• Apply basic prompt engineering and context engineering techniques during testing activities
• Use AI-assisted tools such as Codex, Claude Code, and similar AI command-line interface tools under the guidance of senior team members
• Assist with existing AI evaluation harnesses, test scripts, and regression checks to support quality validation efforts
• Document defects with clear steps to reproduce, expected results, and actual results, and collaborate with engineers, developers, and product owners to investigate and resolve issues
• Contribute to test documentation, reusable test cases, and the continuous improvement of quality assurance processes and standards

About the Team

Our Quality Engineering team is responsible for building reliable, high-quality software and AI-enabled products that our clients and stakeholders depend on. By joining our team, you will be part of exciting work in software testing, AI product evaluation, and quality engineering, while contributing to a collaborative environment that values continuous learning, innovation, and responsible AI adoption. This role offers a meaningful opportunity to develop practical, hands‑on skills at the intersection of traditional quality assurance and emerging artificial intelligence technologies.

For US-based roles only: the anticipated hiring base salary range for this position is$78,300.00-$113,550.00, depending on factors such as experience, education, level, skills, and location. This range is based on a full‑time position. In addition to base salary, this role is eligible for incentive compensation.

Moody’s also offers a competitive benefits package, including not but limited to

• medical
• dental
• vision
• parental leave
• paid time off
• a 401(k) plan with employee and company contribution opportunities
• life, disability, and accident insurance
• a discounted employee stock purchase plan
• tuition reimbursement

Moody’s is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, sex, gender, age, religion or creed, national origin, ancestry, citizenship, marital or familial status,sexual orientation, gender identity, gender expression, genetic information, physical or mental disability, military or veteran status, or any other characteristic protected by law. Moody’s also provides reasonable accommodation to qualified individuals with disabilities or based on a sincerely held religious belief in accordance with applicable laws. If you need to inquire about a reasonable accommodation, or need assistance with completing the application process, please email [e-mail]. This contact information is for accommodation requests only, and cannot be used to inquire about the status of applications

For San Francisco positions, qualified applicants with criminal histories will be considered for employment consistent with the requirements of the San Francisco Fair Chance Ordinance.

This position may be considered a promotional opportunity, pursuant to the Colorado Equal Pay for Equal Work Act.

Candidates for Moody's Corporation may be asked to disclose securities holdings pursuant to Moody’s Policy for Securities Trading and the requirements of the position. Employment is contingent upon compliance with the Policy, including remediation of positions in those holdings as necessary.