Remote AI Testing Consultant - Agent-Based Validation

Trigyn Technologies Limited.; United States; Remote; USD 100,000 - 160,000; Full time

Job summary

Trigyn Technologies Limited. is seeking an AI Testing Consultant to validate AI agents, test workflows, and measure quality, accuracy, and performance across scenarios. The role is remote until noon EST and requires hands-on testing of AI agents, chatbots, or GenAI-based applications.

You will analyze evaluator results, identify gaps and risks, and ensure outputs align with business requirements. Collaboration with onshore teams and developers is essential to support test strategy and defect

Qualifications

• 5+ years in testing/validation roles, preferably AI/ML or intelligent automation.
• Hands-on testing AI agents, chatbots, or GenAI-based apps.
• Understanding of AI evaluation metrics (precision, recall, relevance, groundedness, hallucination).
• Exposure to AI evaluation frameworks like Microsoft AI Foundry.
• Familiarity with prompt-based systems and agent behaviors.

Responsibilities

• Test and validate AI Agents, including single and multi-agent workflows.
• Perform functional, non-functional, and scenario-based testing for AI use cases.
• Use AI evaluators to assess accuracy, relevance, grounding, completeness, and safety.
• Analyze results to identify gaps, risks, and improvements in AI responses.
• Validate outputs against business requirements and expected behavior.
• Collaborate with onshore teams, architects, and developers on test strategy and defect resolution.
• Document test scenarios, results, and evaluation findings clearly.

Skills

AI testing

Agent testing

AI evaluation metrics

Prompt-based systems

Communication & documentation

Tools

Microsoft AI Foundry

Copilot Studio

Job description

Trigyn Technologies Limited. is seeking an AI Testing Consultant to validate AI agents, test workflows, and measure quality, accuracy, and performance across scenarios. The role is remote until noon EST and requires hands-on testing of AI agents, chatbots, or GenAI-based applications.

You will analyze evaluator results, identify gaps and risks, and ensure outputs align with business requirements. Collaboration with onshore teams and developers is essential to support test strategy and defect