AI Quality Engineer: Testing AI-powered Apps

Netrolynx AI; United States; On-site; USD 95,000 - 140,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

Parental leave

Paid time off

401(k) retirement plan with company"s"

Job summary

Moody's Corporation is seeking a Quality Engineer focused on AI-enabled products to join our QA team. You will support quality assurance for software and AI systems, including large language models, ensuring outputs are accurate, relevant, and reliable.

Collaborate with cross-functional teams to develop testing strategies, document findings, and contribute to continuous QA improvements. This role suits someone passionate about AI and ethical AI use, eager to grow hands-on in AI product

Qualifications

• Bachelor's degree in Computer Science, Software Engineering, or a related field.
• Basic understanding of software testing concepts including manual, exploratory, regression, API, and integration testing methodologies.
• Fundamental programming or scripting knowledge, preferably in Python, with familiarity in APIs, databases, JSON, version control, and defect tracking tools.
• Basic understanding of AI concepts, including large language models, with awareness of hallucinations and inconsistent outputs.
• Interest in prompt engineering, context engineering, and evaluating AI-enabled products.
• Strong analytical, communication, documentation, and problem-solving skills, with the ability to collaborate across teams.
• Curiosity and enthusiasm for learning about AI tools and responsible AI practices, including risk management and ethical considerations.

Responsibilities

• Performing manual, exploratory, regression, API, and integration testing within an Agile team environment, under close guidance.
• Testing software features that incorporate AI or large language model components, reviewing outputs for accuracy, relevance, and adherence to instructions.
• Creating basic test scenarios, prompts, and test data to evaluate AI behavior and identify issues such as hallucinations, incomplete responses, or inconsistencies.
• Applying prompt engineering and context engineering techniques during testing to optimize AI performance.
• Utilizing AI-assisted tools like Codex, Claude Code, and similar interfaces under the supervision of senior team members.
• Assisting with existing AI evaluation harnesses, test scripts, and regression checks to validate AI functionalities.
• Documenting defects with clear reproduction steps, expected, and actual results, and collaborating with engineering and product teams to resolve issues.
• Contributing to test documentation, reusable test cases, and ongoing enhancements to QA standards and processes.

Skills

Manual testing

Exploratory testing

Regression testing

API testing

Integration testing

Python

Analytical skills

Communication

Documentation

Problem solving

Cross-team collaboration

AI curiosity

Education

Bachelor's degree in Computer Science, Software Engineering, or a related field

Tools

APIs

Databases

JSON

Version control

Defect tracking tools

Job description

Moody's Corporation is seeking a Quality Engineer focused on AI-enabled products to join our QA team. You will support quality assurance for software and AI systems, including large language models, ensuring outputs are accurate, relevant, and reliable.

Collaborate with cross-functional teams to develop testing strategies, document findings, and contribute to continuous QA improvements. This role suits someone passionate about AI and ethical AI use, eager to grow hands-on in AI product