Senior AI Testing Architect | LLM & RAG Validation Lead

Tata Consultancy Services; Charlotte (NC); On-site; USD 90,000 - 107,000; Full time

Job summary

Tata Consultancy Services in Charlotte, NC seeks an experienced AI testing professional to design end-to-end test plans for generative AI, LLMS, and ML models. The role emphasizes model validation, automation, and data quality to ensure robust, secure AI deployments.

Candidates should have at least 8 years of testing experience, strong Python/PyTest skills, and a background in automation tools. The position focuses on leadership and collaboration with data scientists and software engineers.

Qualifications

• Bachelor’s or master’s degree in computer science or related field.
• Minimum 8 years of experience in testing AI applications and ML systems.
• Experience designing end-to-end test plans for generative AI, LLMs, and ML models.

Responsibilities

• Lead Strategy: Design end-to-end test plans for generative AI, LLMs, and machine learning models.
• Model Validation: Test models for accuracy, drift, latency, robustness, and security vulnerabilities.
• Automation: Build advanced test automation frameworks using Python, PyTest, and specialized AI testing tools.
• Data Quality: Validate training, validation, and test datasets for bias, leakage, and data integrity.
• Performance Testing: Evaluate system scalability and response times under heavy concurrent loads.
• Team Leadership: Mentor junior engineers and collaborate with data scientists and software developers.
• Certifications in Automation

Skills

Test AI applications

RAG systems

agents

APIs

Data Pipeline

Python

PyTest

AI testing tools

Education

Bachelor’s or Master’s degree in computer science or related field

Tools

Python

PyTest

AI testing tools

Job description

Tata Consultancy Services in Charlotte, NC seeks an experienced AI testing professional to design end-to-end test plans for generative AI, LLMS, and ML models. The role emphasizes model validation, automation, and data quality to ensure robust, secure AI deployments.

Candidates should have at least 8 years of testing experience, strong Python/PyTest skills, and a background in automation tools. The position focuses on leadership and collaboration with data scientists and software engineers.