AI Quality & Validation Specialist

jobr.pro; Annandale (VA); Hybrid; USD 60,000 - 80,000; Full time

Benefits offered by this job

Eleven Federal Holidays

Paid Time Off

Parental Leave

Medical plan choices with employer contribution

Dental and Vision Insurance

Company paid Short-Term and Long-Term Disability

401k with competitive matching

Continuing education assistance

Wellness benefits including Calm Health app

Pet Insurance

Job summary

jobr.pro is seeking an AI Quality & Validation Specialist to test and validate applications and AI solutions. This hybrid position is perfect for detail-oriented individuals who can evaluate system behavior and identify any quality issues.

Ideal candidates will have 1-3 years of relevant experience, strong communication skills, and a focus on continuous improvement. The role involves close collaboration with technical teams and project leads, ensuring all outputs meet high-quality standards.

Qualifications

• 1–3 years of relevant experience in software testing or quality assurance.
• Strong attention to detail to identify defects or quality issues.
• Ability to document findings clearly and provide feedback.

Responsibilities

• Support functional, regression, and user-focused testing of applications.
• Evaluate AI-generated outputs for quality and relevance.
• Identify, document, and track software defects and content issues.

Skills

Attention to detail

Documenting findings

Written and verbal communication

End-user perspective

Education

Associate’s / Bachelor’s Degree in a related field

Tools

Defect tracking tools

Agile delivery environments

Job description

Location: Hybrid

Required Clearance: None

Required Education: Associate’s / Bachelor’s Degree

Required Experience: 1–3 years of relevant experience in software testing, quality assurance, AI evaluation, content review, or related technical support

Position Description

PingWind is seeking an AI Quality & Validation Specialist to support the testing, validation, and continuous improvement of applications and AI-enabled solutions prior to release. This junior-level role is ideal for a detail-oriented professional who can evaluate system behavior, identify defects or quality issues, and provide structured feedback to improve application performance and AI-driven outputs. The AI Quality & Validation Specialist will help validate software functionality, assess the quality and relevance of AI-generated results, document findings, and support scenario-based testing from the end-user perspective. This role works closely with technical teams, product stakeholders, and project leads to help ensure released capabilities meet functional requirements and quality standards.

Responsibilities

• Support functional, regression, and user-focused testing of applications and AI-enabled features before release
• Evaluate AI-generated outputs for quality, relevance, accuracy, consistency, and alignment to prompts, instructions, or reference materials
• Identify, document, and track software defects, content issues, edge cases, and output anomalies
• Perform scenario-based testing by interacting with systems as an end user to validate workflows and expected outcomes
• Flag issues in AI or application behavior and provide clear written feedback on what should be corrected or improved
• Assist with validation of prompts, workflows, procedures, and output-review criteria used in AI-supported processes
• Help review text, image, audio, video, or other generated outputs, as applicable to the program
• Maintain test evidence, validation notes, issue logs, and supporting documentation in accordance with project standards
• Collaborate with developers, analysts, engineers, and project leadership to clarify expected behavior and support issue resolution
• Participate in pilot testing of new features, experimental capabilities, and process improvements
• Support continuous quality improvement efforts by identifying recurring issues, trends, and opportunities for better testing coverage
• Contribute to the development of repeatable validation procedures, checklists, and quality review practices

Required Qualifications

• 1–3 years of relevant experience in software testing, QA support, AI output evaluation, digital content review, or an Associate’s / Bachelor’s Degree in a related field
• Strong attention to detail with the ability to identify inconsistencies, defects, or quality issues
• Experience documenting findings clearly and communicating actionable feedback
• Ability to follow structured instructions, evaluation criteria, and testing procedures
• Strong written and verbal communication skills
• Comfortable working across multiple tasks in a fast-paced environment
• Ability to think from the end-user perspective and test systems against expected outcomes
• Basic familiarity with modern software applications, web-based systems, and common collaboration/documentation tools

Desired Qualifications

• Experience evaluating generative AI outputs such as text, images, audio, or video
• Experience supporting software QA, test execution, UAT, or validation activities
• Familiarity with defect tracking tools, test case management, or Agile delivery environments
• Experience comparing outputs against prompts, business rules, or reference materials
• Exposure to prompt testing, AI workflow review, or human-in-the-loop evaluation processes
• Experience in a federal contracting environment
• Active clearance or eligibility to obtain required clearance, if applicable

Benefits

• Eleven Federal Holidays
• Paid Time Off accrued each pay period
• Parental Leave
• Three medical plan choices with generous employer contribution
• Dental and Vision Insurance
• Company paid Short-Term and Long-Term Disability
• Company paid Life and AD&D Insurance
• 401k with competitive matching and vesting schedule
• Continuing education assistance
• Medical, Dependent Care and Commuter Flexible Spending Accounts
• Employee Assistance Program
• Wellness benefits include Calm Health app and WellHub gym subsidy
• 529 College Savings Plan
• Legal Insurance
• Pet Insurance

Veterans are encouraged to apply.

PingWind, Inc. does not discriminate in employment opportunities, terms, and conditions of employment, or practices on the basis of race, age, gender, religious or political beliefs, national origin or heritage, disability, sexual orientation, or any characteristic protected by law.