AI Quality Assurance Engineer

Tekskills Inc.; Bengaluru; On-site; INR 900,000 - 1,300,000; Full time

Job summary

You'll design LLM evaluation strategies, build automated tests, and work with AI engineers to mitigate edge cases and improve prompts. This is a hands-on position for someone who thrives on improving AI systems and metadata quality.

Qualifications

• Strong experience in QA Automation, data quality, or software testing.
• Understanding of AI lifecycle (training vs inference, prompt engineering).
• Proficiency in Python for test automation and data validation.

Responsibilities

• Design and execute LLM evaluation and benchmarking strategies (relevancy, consistency, metadata quality).
• Build and maintain automated testing frameworks using Python.
• Perform prompt regression testing to track impact of changes over time.
• Collaborate with AI engineers to debug edge cases and improve prompts/logic.

Skills

QA Automation

Data Quality

Software Testing

Python

JSON Validation

AWS

Tools

Python

AWS

Job description

Were Hiring: AI Quality Assurance Engineer(MetadataAugmentation)

Location: Pune / Bangalore

Experience: 4+ Years

We are looking for a detail-oriented AI QA Engineer to join our AI Metadata Augmentation team. If you are passionate about ensuring the quality, accuracy, and reliability of AI/LLM outputs, this is a great opportunity to work on cutting-edge AI systems deployed on AWS.

Key Responsibilities:

• Design and execute LLM evaluation & benchmarking strategies (relevancy, consistency, metadata quality)
• Build and maintain automated testing frameworks using Python
• Perform prompt regression testing to track impact of changes over time
• Collaborate with AI engineers to debug edge cases and improve prompts/logic

Required Skills & Experience:

• Strong experience in QA Automation / Data Quality / Software Testing
• Understanding of AI lifecycle (Training vs Inference, Prompt Engineering)
• Proficiency in Python for test automation and data validation
• Hands-on experience with JSON parsing and validation
• Basic knowledge of AWS (CloudWatch, S3)

#Hiring #AIJobs #QAEngineer #MachineLearning #LLM #PythonJobs #AWS #BangaloreJobs #PuneJobs #AIQA