QA Engineer

Siri InfoSolutions Inc; Charlotte (NC); On-site; USD 120,000 - 150,000; Full time

Job summary

Siri InfoSolutions Inc in Charlotte, NC, is seeking a senior QA lead focused on generative AI testing. You will design end-to-end test plans for LLMs and data pipelines, and drive automation using Python and PyTest.

You will validate models for accuracy, drift, latency, and security, mentor junior engineers, and collaborate with data scientists and software developers to ensure data quality and robust performance.

Qualifications

• Bachelor's or master's degree in CS or related field.
• Minimum 8 years of experience.
• Experience leading AI testing, ML models, and QA for generative AI.
• Certifications in Automation.

Responsibilities

• Lead end-to-end test plans for generative AI, LLMs, and ML models.
• Model validation for accuracy, drift, latency, robustness, and security vulnerabilities.
• Automation: Build advanced test automation frameworks using Python, PyTest, and specialized AI testing tools.
• Data quality: Validate training, validation, and test datasets for bias, leakage, and data integrity.
• Performance testing: Evaluate system scalability and response times under heavy concurrent loads.
• Team leadership: Mentor junior engineers and collaborate with data scientists and software developers.
• Certifications in Automation.

Skills

Test AI applications

RAG systems

agents

APIs

Data Pipeline

Automated testing frameworks

Education

Bachelor's or master's degree in CS or related field

Tools

Python

PyTest

AI testing tools

Job description

Primary Skill: Test AI applications, RAG systems, agents, APIs and Data Pipeline using automated development framework

Experience: Minimum 8 years

Roles & Responsibilities

Bachelor's or master's degree in computer science or related field.

Key Responsibilities

• Lead Strategy: Design end-to-end test plans for generative AI, LLMs, and machine learning models.
• Model Validation: Test models for accuracy, drift, latency, robustness, and security vulnerabilities.
• Automation: Build advanced test automation frameworks using Python, PyTest, and specialized AI testing tools.
• Data Quality: Validate training, validation, and test datasets for bias, leakage, and data integrity.
• Performance Testing: Evaluate system scalability and response times under heavy concurrent loads.
• Team Leadership: Mentor junior engineers and collaborate with data scientists and software developers.
• Certifications in Automation