Functional AI Tester - GenAI

Michelin; Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Michelin in Pune is seeking a Functional AI Tester - GenAI to join the QA team. You will design and execute end-to-end test strategies for GenAI features, focusing on reliability, safety and governance, with emphasis on Python-based automation.

You will evaluate RAG, conversational AI, and agentic components, perform ETL and data quality validation, and collaborate with AI engineers to define measurable acceptance criteria and ship high-quality AI features.

Responsibilities

• Test strategy and planning
• Python test automation
• GenAI evaluation
• RAG and semantic retrieval testing
• API and application testing
• ETL and data quality validation
• Non-functional testing
• Share results and insights; recommend remediation and preventive actions.

Job description

Functional AI Tester - GenAI

We are seeking a Quality Assurance (QA) Engineer focused on testing Generative AI (GenAI) applications with a strong emphasis on Python-based test automation, GenAI evaluation, and ETL/data quality validation. You will design and execute end-to-end test strategies that ensure our AI solutions are accurate, reliable, safe, and compliant.

About the Role

You will be involved in QA for GenAI features including Retrieval-Augmented Generation (RAG), conversational AI and Agentic evaluations. The role centers on:

• Systematic GenAI evaluation (qualitative and quantitative metrics)




• ETL and data quality testing for the data flows that feed AI systems




• Python-driven automated testing

This position is hands-on and collaborative, partnering with AI engineers, data engineers, and product teams to define measurable acceptance criteria and ship high-quality AI features.

Key Responsibilities

• Test strategy and planning




• Python test automation




• GenAI evaluation




• RAG and semantic retrieval testing




• API and application testing




• ETL and data quality validation




• Non-functional testing




• Share results and insights; recommend remediation and preventive actions.

Required Qualifications

• Experience




• Technical skills




• Quality and governance




• Soft skills

Nice to Have

• Experience with evaluation frameworks or tooling for LLMs and RAG quality measurement.

Experience creating synthetic datasets to stress specific behaviors.