Test Analyst - AI Assurance

Hexaware Technologies; Chennai District; On-site; INR 800,000 - 1,100,000; Full time

Job summary

Hexaware Technologies is seeking a Test Analyst specialized in AI Assurance to shape the AI quality and safety checks across GenAI systems. The role focuses on designing evaluation strategies, integrating with CI/CD, and leading a technical team.

The ideal candidate has expert Python skills and strong experience with AI evaluation tools and observability platforms to ensure responsible AI behavior.

Qualifications

• Experience designing AI assurance strategies for GenAI systems.
• Ability to build evaluation frameworks for quality, safety and reliability.
• Proficiency in Python and CI/CD integration for AI pipelines.

Responsibilities

• Develop AI Assurance strategy to assess AI systems for quality, safety, reliability and responsible AI behavior.
• Design & implement AI evaluation framework using Ragas, DeepEval, etc.
• Integrate AI evaluation with CI/CD pipelines for automated checks.
• Lead and mentor a team to execute AI assurance initiatives.

Skills

GenAI architecture

RAG and Agents

Python

AI evaluation

CI/CD integration

Team management

Tools

Arize

Langfuse

Haize Labs

Galileo

Okareo

LangSmith

Job description

Test Analyst - AI Assurance

• Expert understanding of GenAI system architecture (RAG / Agents)
• Develop AI Assurance strategy to assess AI systems for quality, safety, reliability and responsible AI behavior
• Design & implement AI evaluation framework using Ragas, DeepEval,etc
• Expert level tool expertise on various market popular AI Evaluations tools & AI Observability platforms like Arize, Langfuse, Haize Labs, Galileo, Okareo, LangSmith, etc
• Integrate AI evaluation with CI/CD pipeline
• Team Management skills.
• Expert in Python.