QA Engineer with AI testing

VOLTO Consulting; Charlotte (NC); On-site; USD 120,000 - 180,000; Full time

Job summary

VOLTO Consulting in Charlotte, NC seeks a senior QA specialist to lead testing of AI applications, RAG systems, agents, APIs and data pipelines using an automated development framework. This onsite, full-time role requires strong expertise in automated testing and ML workflows.

You will design end-to-end test plans for generative AI and ML models, validate data quality, monitor performance under load, and mentor junior engineers while collaborating with data scientists and software developers.

Qualifications

• Bachelor's or master's degree in computer science or related field.
• Minimum 8 years of experience in software testing or AI testing domains.

Responsibilities

• Lead Strategy: Design end-to-end test plans for generative AI, LLMs, and machine learning models.
• Model Validation: Test models for accuracy, drift, latency, robustness, and security vulnerabilities.
• Automation: Build advanced test automation frameworks using Python, PyTest, and specialized AI testing tools.
• Data Quality: Validate training, validation, and test datasets for bias, leakage, and data integrity.
• Performance Testing: Evaluate system scalability and response times under heavy concurrent loads.
• Team Leadership: Mentor junior engineers and collaborate with data scientists and software developers.
• Certifications in Automation

Skills

Test AI applications

RAG systems

agents

APIs

Data Pipeline

Automated development framework

Education

Bachelor's or Master's in Computer Science or related field

Tools

Python

PyTest

Job description

Must Have Technical/Functional Skills

Primary Skill: Test AI applications, RAG systems, agents, APIs and Data Pipeline using automated development framework

Charlotte, NC - Onsite

Full-time

Job Description

Must Have Technical/Functional Skills

Primary Skill: Test AI applications, RAG systems, agents, APIs and Data Pipeline using automated development framework

Experience: Minimum 8 years

Roles & Responsibilities

• Bachelor s or master s degree in computer science or related field.

Key Responsibilities

• Lead Strategy: Design end-to-end test plans for generative AI, LLMs, and machine learning models.
• Model Validation: Test models for accuracy, drift, latency, robustness, and security vulnerabilities.
• Automation: Build advanced test automation frameworks using Python, PyTest, and specialized AI testing tools.
• Data Quality: Validate training, validation, and test datasets for bias, leakage, and data integrity.
• Performance Testing: Evaluate system scalability and response times under heavy concurrent loads.
• Team Leadership: Mentor junior engineers and collaborate with data scientists and software developers.
• Certifications in Automation