AI Automation Testing

Hexaware Technologies; Chennai District; On-site; INR 1,800,000 - 2,200,000; Full time

Job summary

Hexaware Technologies in Chennai region seeks an expert to lead GenAI architecture and AI assurance initiatives. The role focuses on designing evaluation frameworks, integrating tools such as Arize and LangSmith, and ensuring responsible AI behavior across systems.

You will drive implementation of RAG/Agents based architectures, connect AI evaluation with CI/CD, and manage a high-performing team delivering reliable AI solutions.

Qualifications

• Expertise in GenAI system architecture including retrieval-augmented generation (RAG) and agents.
• Design and implement AI evaluation frameworks and quality checks.
• Proficiency with market AI evaluation tools and AI observability platforms.
• Experience integrating AI evaluation into CI/CD pipelines.
• Proven team management and delivery of AI projects.

Responsibilities

• Develop AI Assurance strategy to assess AI systems for quality, safety, reliability and responsible AI behavior.
• Design & implement AI evaluation framework using RAGs, DeepEval, etc.
• Operate with expert toolsets for AI evaluation and observability tools like Arize, Langfuse, Haize Labs, Galileo, Okareo, LangSmith, etc.
• Integrate AI evaluation into CI/CD pipelines.
• Lead and manage the AI/GeneAI team.

Skills

GenAI architecture

RAG/Agents

AI evaluation tools

Observability platforms

CI/CD integration

Team management

Tools

Arize

Langfuse

Haize Labs

Galileo

Okareo

LangSmith

Job description

Expert understanding of GenAI system architecture (RAG / Agents)

• Develop AI Assurance strategy to assess AI systems for quality, safety, reliability and responsible AI behavior
• Design & implement AI evaluation framework using Ragas, DeepEval, etc
• Expert level tool expertise on various market popular AI Evaluations tools & AI Observability platforms like Arize, Langfuse, Haize Labs, Galileo, Okareo, LangSmith, etc
• Integrate AI evaluation with CI/CD pipeline
• Team Management skills

Expert in Pytho