AI Quality Engineer

PhoenixTeam; Arlington (VA); Remote; USD 110,000 - 160,000; Full time

Job summary

PhoenixTeam is seeking an AI Quality Engineer to design, validate, and integrate AI-enabled solutions for federal modernization efforts. You will establish QA frameworks, test AI-generated code and prototypes, and collaborate with DevOps on multi‑agent workflows to ensure security, compliance, and production readiness.

The role requires hands‑on testing of AI outputs, experience with CI/CD, and strong Communication as you coordinate with product, engineering, and business teams.

Qualifications

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent professional experience.
• Demonstrated experience in software quality assurance, test engineering, or a related discipline, including test plan design and defect management.
• Hands‑on experience testing or validating AI‑generated code, prototypes, or “vibe‑coded” solutions for accuracy and reliability.
• Experience with automated testing tools and frameworks (e.g., Selenium) across web and enterprise applications.
• Experience with one or more of the following: Salesforce, AWS (including Lambda and Redshift), Java, or PostgreSQL, or a comparable enterprise technology stack.
• Familiarity with CI/CD pipelines and quality gates within modern software delivery practices.
• Strong analytical skills, with the ability to identify security, data‑sensitivity, and compliance risks in AI‑assisted solutions.
• Excellent written and verbal communication skills, with the ability to work across product, engineering, data, and business SME teams.
• Must be a U.S. citizen and eligible to obtain and maintain a Public Trust clearance / favorable suitability determination.

Responsibilities

• Design and implement AI‑enabled solutions for customer modernization use cases, developing and integrating AI capabilities into existing systems and workflows.
• Establish and enforce a quality assurance framework — test plans, acceptance criteria, and regression suites — for validating AI‑generated code, skills, and agents before they are promoted through the innovation pipeline.
• Guide the development of reusable AI skills and agents for customer modernization tasks, with emphasis on automated testing, security scanning, data sensitivity, and scalability.
• Collaborate with DevOps engineers to create and orchestrate long‑running, multi‑agent workflows, ensuring reliability, observability, and recoverability in production environments.
• Support creation of the innovation pipeline that moves AI‑generated prototypes from sandbox concepts into secure, enterprise‑grade applications.
• Support Value Engineers’ efforts by reviewing and quality‑checking prototype outputs for accuracy, reliability, and adherence to standards prior to pipeline promotion.
• Continuously monitor AI‑assisted development outputs for accuracy, reliability, and compliance drift, feeding findings back to improve prompts, skills, and agents over time.

Skills

QA testing

AI validation

CI/CD pipelines

Selenium

Security risk assessment

Analytical skills

Excellent communication

US citizenship

Education

Bachelor's degree or equivalent experience

Tools

Salesforce

AWS

Java

PostgreSQL

Redshift

Lambda

Job description

The AI Quality Engineer supports a federal customer by helping design and implement AI-enabled solutions, developing and integrating AI capabilities, and ensuring that skills, agents, and AI-generated prototypes meet quality, security, and compliance standards as they move toward production. This work supports broader modernization goals, including automating and streamlining servicing operations, reducing manual processing, improving data quality, aligning with the customer’s modernization standards, and establishing a roadmap toward the eventual retirement of legacy platforms. Success requires a bachelor’s degree (or equivalent experience), demonstrated experience in software quality assurance and test engineering, hands‑on experience validating AI-generated code and prototypes, familiarity with CI/CD pipelines and quality gates, strong analytical skills, and excellent written and verbal communication.

Key Functions

• Design and implement AI‑enabled solutions for customer modernization use cases, developing and integrating AI capabilities into existing systems and workflows.
• Establish and enforce a quality assurance framework — test plans, acceptance criteria, and regression suites — for validating AI‑generated code, skills, and agents before they are promoted through the innovation pipeline.
• Guide the development of reusable AI skills and agents for customer modernization tasks, with particular emphasis on automated testing, security scanning, data sensitivity, and scalability.
• Collaborate with DevOps engineers to create and orchestrate long‑running, multi‑agent workflows, ensuring reliability, observability, and recoverability in production environments.
• Support creation of the innovation pipeline that moves AI‑generated (“vibe‑coded”) prototypes from sandbox concepts into secure, enterprise‑grade applications.
• Support Value Engineers’ efforts by reviewing and quality‑checking prototype outputs for accuracy, reliability, and adherence to standards prior to pipeline promotion.
• Continuously monitor AI‑assisted development outputs for accuracy, reliability, and compliance drift, feeding findings back to Value Engineers and AI Engineers to improve prompts, skills, and agents over time.

Minimum Qualifications

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent professional experience.
• Demonstrated experience in software quality assurance, test engineering, or a related discipline, including test plan design and defect management.
• Hands‑on experience testing or validating AI‑generated code, prototypes, or “vibe‑coded” solutions for accuracy and reliability.
• Experience with automated testing tools and frameworks (e.g., Selenium) across web and enterprise applications.
• Experience with one or more of the following: Salesforce, AWS (including Lambda and Redshift), Java, or PostgreSQL, or a comparable enterprise technology stack.
• Familiarity with CI/CD pipelines and quality gates within modern software delivery practices.
• Strong analytical skills, with the ability to identify security, data‑sensitivity, and compliance risks in AI‑assisted solutions.
• Excellent written and verbal communication skills, with the ability to work across product, engineering, data, and business SME teams.
• Must be a U.S. citizen and eligible to obtain and maintain a Public Trust clearance / favorable suitability determination.

Preferred Qualifications or Skills

• Direct experience supporting Federal clients.
• Experience with FedRAMP authorization processes.
• Mortgage, fintech, or other regulated industry experience (preferred, not required — willingness to learn quickly is essential).
• Familiarity with AI safety, governance, and compliance.
• Ability to use or teach tools such as ChatGPT, Claude, Gemini, Notion, Airtable, Zapier, LangChain, or similar.
• Experience in startups, consulting, internal strategy, learning & development (L&D), or innovation teams.

Position Details

• Employment type: Full‑Time
• Location: 100% Remote (US‑based)
• Hours: 8am–5pm ET
• Eligibility: Must be able to obtain and maintain a Public Trust security clearance

About PhoenixTeam

PhoenixTeam partners with organizations to deliver real results across GenAI, data, product, and technology. We help clients modernize legacy systems, redesign complex workflows, and move practical AI and digital solutions into production in regulated environments.

PhoenixTeam is an Equal Opportunity Employer. We consider qualified applicants without regard to race, color, religion, sex, national origin, age, disability, genetic information, protected veteran status, or any other status protected by applicable law.

Equal employment opportunity, including veterans and individuals with disabilities.