AI Quality Assurance Engineer

Tekskills Inc.; Hyderabad; On-site; INR 800,000 - 1,100,000; Full time

Job summary

Tekskills Inc. is hiring an AI Quality Assurance Engineer (MetadataAugmentation) to join our AI Metadata Augmentation team. Location options include Pune or Bangalore, with 4+ years of relevant experience expected.

The role focuses on ensuring quality, accuracy, and reliability of AI/LLM outputs deployed on AWS. You will design evaluation strategies, build automated tests in Python, and collaborate to improve prompts and logic.

Qualifications

• 4+ years of experience in QA automation or data quality
• Understanding of AI lifecycle (training vs inference, prompt engineering)
• Proficiency in Python for test automation and data validation
• Hands-on experience with JSON parsing and validation
• Basic knowledge of AWS services (CloudWatch, S3)

Responsibilities

• Design and execute LLM evaluation & benchmarking strategies (relevancy, consistency, metadata quality)
• Build and maintain automated testing frameworks using Python
• Perform prompt regression testing to track impact of changes over time
• Collaborate with AI engineers to debug edge cases and improve prompts/logic

Skills

QA Automation

Data Quality

Software Testing

AI lifecycle

Python

JSON parsing

AWS

Tools

Python

AWS

JSON

Job description

Were Hiring: AI Quality Assurance Engineer(MetadataAugmentation)

Location: Pune / Bangalore
Experience: 4+ Years

We are looking for a detail-oriented AI QA Engineer to join our AI Metadata Augmentation team. If you are passionate about ensuring the quality, accuracy, and reliability of AI/LLM outputs, this is a great opportunity to work on cutting-edge AI systems deployed on AWS.

Key Responsibilities:

• Design and execute LLM evaluation & benchmarking strategies (relevancy, consistency, metadata quality)
• Build and maintain automated testing frameworks using Python
• Perform prompt regression testing to track impact of changes over time
• Collaborate with AI engineers to debug edge cases and improve prompts/logic

Required Skills & Experience:

• Strong experience in QA Automation / Data Quality / Software Testing
• Understanding of AI lifecycle (Training vs Inference, Prompt Engineering)
• Proficiency in Python for test automation and data validation
• Hands-on experience with JSON parsing and validation
• Basic knowledge of AWS (CloudWatch, S3)

#Hiring #AIJobs #QAEngineer #MachineLearning #LLM #PythonJobs #AWS #BangaloreJobs #PuneJobs #AIQA