QA AI Lead

Omegahires; United States; On-site; USD 120,000 - 200,000; Full time

Job summary

Omegahires is seeking a QA AI Lead to define and implement enterprise-wide AI testing and quality assurance strategies across AI/ML and GenAI applications, from development to production deployment.

You will mentor QA engineers, establish testing standards, integrate automated frameworks into CI/CD, and collaborate with product, data science, and security teams to ensure responsible AI, performance, and compliance.

Qualifications

• Bachelor's degree in CS/IT/Engineering; master's preferred.
• 10+ years in Software Quality Assurance with leadership experience.
• 3+ years testing AI/ML or Generative AI solutions; strong automation skills.

Responsibilities

• Define and implement enterprise-wide AI testing and quality assurance strategies.
• Establish testing standards, governance, and best practices for AI/ML and Generative AI applications.
• Lead QA activities across AI product lifecycle from development through production deployment.
• Drive AI-powered quality engineering transformation initiatives.
• Validate LLMs, Chatbots, Copilots, and GenAI applications.
• Develop test scenarios for prompt validation, hallucination detection, bias and fairness testing, security and privacy testing, performance and scalability testing, and explainability validation.
• Define AI model quality metrics and monitoring standards.
• Design and implement automated testing frameworks.
• Integrate automated testing into CI/CD pipelines.
• Establish AI-assisted test generation and test optimization strategies.
• Drive shift-left and continuous testing practices.
• Lead and mentor QA Engineers, Automation Engineers, and AI Test Specialists.
• Collaborate with Product Owners, Architects, Developers, Data Scientists, and Business Analysts.
• Manage vendor and offshore QA teams.
• Conduct test planning, effort estimation, and release readiness reviews.
• Monitor quality KPIs and AI performance metrics.
• Present quality status and risk assessments to leadership teams.
• Ensure compliance with security, regulatory, and Responsible AI requirements.

Skills

QA Leadership

Test Automation

AI/ML testing

Team mentoring

Education

Bachelor's degree in CS/IT/Engineering

Master's degree preferred

Tools

CI/CD pipelines

Automation frameworks

AI testing tools

Job description

QA AI Lead Responsibilities

• Define and implement enterprise-wide AI testing and quality assurance strategies.
• Establish testing standards, governance, and best practices for AI/ML and Generative AI applications.
• Lead QA activities across AI product lifecycle from development through production deployment.
• Drive AI-powered quality engineering transformation initiatives.
• Validate LLMs, Chatbots, Copilots, and GenAI applications.
• Develop test scenarios for prompt validation, hallucination detection, bias and fairness testing, security and privacy testing, performance and scalability testing, and explainability validation.
• Define AI model quality metrics and monitoring standards.
• Design and implement automated testing frameworks.
• Integrate automated testing into CI/CD pipelines.
• Establish AI-assisted test generation and test optimization strategies.
• Drive shift-left and continuous testing practices.
• Lead and mentor QA Engineers, Automation Engineers, and AI Test Specialists.
• Collaborate with Product Owners, Architects, Developers, Data Scientists, and Business Analysts.
• Manage vendor and offshore QA teams.
• Conduct test planning, effort estimation, and release readiness reviews.
• Monitor quality KPIs and AI performance metrics.
• Present quality status and risk assessments to leadership teams.
• Ensure compliance with security, regulatory, and Responsible AI requirements.

Required Skills

• Bachelor's degree in Computer Science, Information Technology, Engineering, or related field; Master's degree preferred.
• 10+ years of experience in Software Quality Assurance.
• 5+ years of Test Automation experience.
• 3+ years of experience testing AI/ML or Generative AI solutions.
• Experience leading QA teams and enterprise testing programs.