AI Quality Assurance Engineer

Tekskills Inc.; Kolkata District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Tekskills Inc. in India is hiring an AI Quality Assurance Engineer (MetadataAugmentation) to ensure quality, accuracy, and reliability of AI outputs. The role focuses on evaluating LLMs, building tests, and collaborating with AI engineers.

You will design benchmarking strategies, automate tests in Python, test prompt changes, and validate JSON data while monitoring AWS services like CloudWatch and S3.

Qualifications

• Experience in QA automation, data quality, or software testing.
• Understanding of AI lifecycle (training vs inference, prompt engineering).
• Proficiency in Python for test automation and data validation.
• Hands-on experience with JSON parsing and validation.
• Basic knowledge of AWS (CloudWatch, S3).

Responsibilities

• Design and execute LLM evaluation & benchmarking strategies (relevancy, consistency, metadata quality).
• Build and maintain automated testing frameworks using Python.
• Perform prompt regression testing to track impact of changes over time.
• Collaborate with AI engineers to debug edge cases and improve prompts/logic.

Skills

QA Automation

Data Quality

Software Testing

AI lifecycle understanding

Python

JSON parsing

AWS basics

Job description

Were Hiring: AI Quality Assurance Engineer(MetadataAugmentation)

Location: Pune / Bangalore
Experience: 4+ Years

We are looking for a detail-oriented AI QA Engineer to join our AI Metadata Augmentation team. If you are passionate about ensuring the quality, accuracy, and reliability of AI/LLM outputs, this is a great opportunity to work on cutting-edge AI systems deployed on AWS.

Key Responsibilities:

• Design and execute LLM evaluation & benchmarking strategies (relevancy, consistency, metadata quality)
• Build and maintain automated testing frameworks using Python
• Perform prompt regression testing to track impact of changes over time
• Collaborate with AI engineers to debug edge cases and improve prompts/logic

Required Skills & Experience:

• Strong experience in QA Automation / Data Quality / Software Testing
• Understanding of AI lifecycle (Training vs Inference, Prompt Engineering)
• Proficiency in Python for test automation and data validation
• Hands-on experience with JSON parsing and validation
• Basic knowledge of AWS (CloudWatch, S3)

#Hiring #AIJobs #QAEngineer #MachineLearning #LLM #PythonJobs #AWS #BangaloreJobs #PuneJobs #AIQA