AI Quality Assurance Engineer

Tekskills Inc.; Mumbai; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Tekskills Inc. is hiring an AI Quality Assurance Engineer (MetadataAugmentation) to join the AI Metadata Augmentation team in Pune/Bangalore. The role focuses on ensuring quality, accuracy, and reliability of AI/LLM outputs on AWS, with 4+ years of experience targeted.

The candidate will design LLM evaluation plans, build automated testing frameworks in Python, and perform prompt regression testing while collaborating to debug edge cases and improve prompts and logic.

Qualifications

• Strong experience in QA Automation / Data Quality / Software Testing.
• Understanding of AI lifecycle (Training vs Inference, Prompt Engineering).
• Proficiency in Python for test automation and data validation.
• Hands-on experience with JSON parsing and validation.
• Basic knowledge of AWS (CloudWatch, S3).

Responsibilities

• Design and execute LLM evaluation & benchmarking strategies.
• Build and maintain automated testing frameworks using Python.
• Perform prompt regression testing to track impact of changes over time.
• Collaborate with AI engineers to debug edge cases and improve prompts/logic.

Skills

QA Automation

Data Quality

Software Testing

Python

JSON parsing

AWS

Job description

Were Hiring: AI Quality Assurance Engineer(MetadataAugmentation)

Location: Pune / Bangalore

Experience: 4+ Years

We are looking for a detail-oriented AI QA Engineer to join our AI Metadata Augmentation team. If you are passionate about ensuring the quality, accuracy, and reliability of AI/LLM outputs, this is a great opportunity to work on cutting-edge AI systems deployed on AWS.

Key Responsibilities:

• Design and execute LLM evaluation & benchmarking strategies (relevancy, consistency, metadata quality)
• Build and maintain automated testing frameworks using Python
• Perform prompt regression testing to track impact of changes over time
• Collaborate with AI engineers to debug edge cases and improve prompts/logic

Required Skills & Experience:

• Strong experience in QA Automation / Data Quality / Software Testing
• Understanding of AI lifecycle (Training vs Inference, Prompt Engineering)
• Proficiency in Python for test automation and data validation
• Hands-on experience with JSON parsing and validation
• Basic knowledge of AWS (CloudWatch, S3)

#Hiring #AIJobs #QAEngineer #MachineLearning #LLM #PythonJobs #AWS #BangaloreJobs #PuneJobs #AIQA