AI Tester

SIRO Clinpharm Pvt. Ltd.; United States; Remote; USD 90,000 - 130,000; Full time

Job summary

SIRO Clinpharm Pvt. Ltd. seeks an AI Tester to ensure QA for AI/ML web apps, with emphasis on performance, accuracy, security and regulatory compliance. The role covers prompt engineering, RAG validation and AI observability in a remote setup.

Responsibilities include designing tests, creating plans, collaborating across teams, and validating model quality and drift in GxP-regulated environments.

Qualifications

• Extensive QA/testing or validation experience in AI/ML systems.
• Solid understanding of LLMs, evaluation metrics and model drift.
• Experience in API testing, SQL/data validation and UI testing.
• Hands-on testing with GenAI/LLM, prompt engineering, and RAG validation.
• Knowledge of Responsible AI, bias, fairness and output reliability.
• Python or TypeScript for test automation.
• Experience analyzing logs with log aggregation and AI observability tools.
• Exposure to GxP-regulated environments in pharma/biotech.

Responsibilities

• Design and execute tests for AI/ML web apps, covering edge cases and prompts.
• Create test plans, strategies and detailed test cases.
• Use defect tracking and test management tools for testing activities.
• Collaborate with product, development, business and platform teams.
• Test AI chat features and standard UI components.
• Validate model quality, metrics, drift, and performance.
• Test RAG pipelines, retrieval accuracy and grounding.
• Conduct non-functional testing including performance and security.
• Support validation evidence, audit/compliance activities.
• Perform AI observability testing with logs and monitoring.

Skills

QA/testing

AI concepts & LLMs

API testing

SQL / Data validation

Test design

Python

TypeScript

AI observability

RAG validation

Hallucination testing

Responsible AI practices

GxP/pharma validation

Communication skills

Problem-solving

Tools

Log analysis tools

Test automation tooling

Job description

Job Title: AI Tester

Location: Pan India (Remote Work Mode)

Experience: 512 Years

Role Overview

Responsible for QA and testing of AI/ML and LLM-based web applications, ensuring performance, reliability, accuracy, security, and regulatory compliance.

Key Responsibilities

• Design and execute functional tests covering edge cases, prompt variations, bias validation, and LLM behavior.
• Create comprehensive test plans, test strategies, and test cases.
• Use bug-tracking and test case management tools to manage defects and testing activities.
• Collaborate with product, development, business, and platform teams.
• Test AI chat features, along with standard website and UI components.
• Validate model quality, evaluation metrics, model drift, and performance.
• Test RAG pipelines, retrieval accuracy, response grounding, hallucinations, and agent/tool-use scenarios.
• Conduct non-functional testing, including performance, reliability, and security testing.
• Support test documentation, validation evidence, and audit/compliance activities.
• Perform AI observability testing, including application logs, monitoring, and model behavior analysis.

Requirements

• 37 years of experience in QA/testing or validation, preferably involving AI/ML systems.
• Good understanding of AI concepts, LLMs, evaluation metrics, and model drift.
• Experience in API testing, SQL/data validation, website/UI testing, and test design.
• Hands-on experience with GenAI/LLM testing, prompt engineering, RAG validation, and hallucination testing.
• Knowledge of Responsible AI practices, including bias, fairness, and output reliability.
• Knowledge of Python or TypeScript for test automation.
• Experience analyzing application logs using log aggregation and AI observability tools.
• Exposure to GxP-regulated environments, preferably within pharmaceutical, biotechnology, or life sciences organizations.
• Strong analytical, problem-solving, and communication skills.
• Hands-on experience in LLM/RAG validation, hallucination testing, model drift detection, Responsible AI practices, and AI observability.
• Experience in GxP/pharma validation, model drift testing, Python-based test automation, and AI observability.