AI Quality Assurance Engineer

Tekskills Inc.; Chennai; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

Tekskills Inc. in Pune/Bangalore is hiring a detail-oriented AI QA Engineer to join the AI Metadata Augmentation team. You will ensure quality, accuracy, and reliability of AI/LLM outputs deployed on AWS.

Responsibilities include designing LLM evaluation and benchmarking strategies, building automated testing frameworks in Python, and performing prompt regression testing to track changes over time. Collaboration with AI engineers to debug edge cases and improve prompts.

Qualifications

• Strong background in QA Automation / Data Quality / Software Testing.
• Understanding of AI lifecycle (Training vs Inference, Prompt Engineering).
• Proficiency in Python for test automation and data validation.
• Hands-on experience with JSON parsing and validation.
• Basic knowledge of AWS (CloudWatch, S3).

Responsibilities

• Design and execute LLM evaluation & benchmarking strategies (relevancy, consistency, metadata quality).
• Build and maintain automated testing frameworks using Python.
• Perform prompt regression testing to track impact of changes over time.
• Collaborate with AI engineers to debug edge cases and improve prompts/logic.

Skills

QA Automation

Python

JSON parsing

AWS

Tools

AWS

Job description

Were Hiring: AI Quality Assurance Engineer(MetadataAugmentation)

Location: Pune / Bangalore

Experience: 4+ Years

We are looking for a detail-oriented AI QA Engineer to join our AI Metadata Augmentation team. If you are passionate about ensuring the quality, accuracy, and reliability of AI/LLM outputs, this is a great opportunity to work on cutting-edge AI systems deployed on AWS.

Key Responsibilities:

• Design and execute LLM evaluation & benchmarking strategies (relevancy, consistency, metadata quality)
• Build and maintain automated testing frameworks using Python
• Perform prompt regression testing to track impact of changes over time
• Collaborate with AI engineers to debug edge cases and improve prompts/logic

Required Skills & Experience:

• Strong experience in QA Automation / Data Quality / Software Testing
• Understanding of AI lifecycle (Training vs Inference, Prompt Engineering)
• Proficiency in Python for test automation and data validation
• Hands-on experience with JSON parsing and validation
• Basic knowledge of AWS (CloudWatch, S3)

#Hiring #AIJobs #QAEngineer #MachineLearning #LLM #PythonJobs #AWS #BangaloreJobs #PuneJobs #AIQA