AI Quality Assurance Engineer

Tekskills; Bengaluru; On-site; INR 800,000 - 1,100,000; Full time

Job summary

Tekskills is seeking a detail-oriented AI QA Engineer to join our AI Metadata Augmentation team in Bengaluru. You will drive LLM evaluation, build automated tests in Python, and ensure the quality of AI outputs on AWS infrastructure.

You will collaborate with AI engineers to identify edge cases, perform regression testing, and help improve prompts and logic for reliable AI systems.

Qualifications

• Strong experience in QA Automation / Data Quality / Software Testing.
• Understanding of AI lifecycle (Training vs Inference, Prompt Engineering).
• Proficiency in Python for test automation and data validation.
• Hands-on experience with JSON parsing and validation.
• Basic knowledge of AWS (CloudWatch, S3).

Responsibilities

• Design and execute LLM evaluation and benchmarking for relevancy and metadata quality.
• Build and maintain automated testing frameworks using Python.
• Perform prompt regression testing to track impact of changes over time.
• Collaborate with AI engineers to debug edge cases and improve prompts/logic.

Skills

QA Automation

Data Quality

Software Testing

AI lifecycle

JSON parsing

Cloud basics

Tools

Python

AWS (CloudWatch, S3)

Job description

We are looking for a detail-oriented AI QA Engineer to join our AI Metadata Augmentation team. If you are passionate about ensuring the quality, accuracy, and reliability of AI/LLM outputs, this is a great opportunity to work on cutting-edge AI systems deployed on AWS.

Key Responsibilities

• Design and execute LLM evaluation & benchmarking strategies (relevancy, consistency, metadata quality)
• Build and maintain automated testing frameworks using Python
• Perform prompt regression testing to track impact of changes over time
• Collaborate with AI engineers to debug edge cases and improve prompts/logic

Required Skills & Experience

• Strong experience in QA Automation / Data Quality / Software Testing
• Understanding of AI lifecycle (Training vs Inference, Prompt Engineering)
• Proficiency in Python for test automation and data validation
• Hands-on experience with JSON parsing and validation
• Basic knowledge of AWS (CloudWatch, S3)