Ai Automation Cgi Chennai Bengaluru

vibehackers; Chennai District; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

vibehackers in Chennai, India is seeking an Automation Quality Engineer to design, develop, and maintain automation test frameworks in Java and/or Python for AI-enabled applications, including LLM-based systems, RAG pipelines, and vector databases.

You will validate prompts, test embeddings, and integrate with CI/CD, collaborating with data scientists and developers to ensure quality, performance, and security.

Qualifications

• Strong programming skills in Python and/or Java.
• Hands-on experience with automation testing frameworks (Selenium, Playwright, REST Assured, PyTest).
• Practical experience with LLMs (OpenAI GPT, Azure OpenAI, or similar).
• Solid understanding and exposure to RAG architectures.
• Experience working with vector databases for embeddings and semantic search (FAISS, Pinecone, Chroma, Weaviate).
• Hands-on usage of OpenAI APIs (chat/completions, embeddings, etc.).
• Experience with LangChain for prompt chaining, memory, and tool usage.
• Strong knowledge of API testing, SQL, and data validation.
• Familiarity with CI/CD tools such as Jenkins, GitHub Actions, Azure DevOps.
• Good understanding of SDLC, STLC, and Agile methodologies.

Responsibilities

• Design, develop, and maintain automation test frameworks using Java and/or Python.
• Implement UI, API, and backend automation using tools such as Selenium, Playwright, REST Assured, PyTest, or similar.
• Validate LLM-based applications for prompt accuracy, response quality, hallucination detection, and consistency.
• Test and evaluate RAG pipelines, including document ingestion, embeddings, retrieval accuracy, and response relevance.
• Work with vector databases (FAISS, Pinecone, Chroma, Weaviate) to validate indexing, retrieval, and performance.
• Integrate and test applications using OpenAI APIs and similar LLM providers.
• Leverage LangChain (or equivalent frameworks) to test chains, agents, tools, and workflows.
• Create automated test cases for AI workflows, model outputs, prompt versions, and inference pipelines.
• Implement data-driven and AI-assisted testing strategies and participate in CI/CD pipelines to enforce quality gates.
• Collaborate with Data Scientists, Developers, and Product teams to define quality metrics and ensure compliance with security, performance, and scalability standards.

Skills

Python

Java

Selenium

Playwright

REST Assured

PyTest

OpenAI APIs

LangChain

API Testing

SQL

CI/CD

Security Testing

Tools

Selenium

Playwright

REST Assured

PyTest

Job description

Heavy focus on testing LLMs, RAG, vector DBs and LangChain—so it's directly about working with AI dev tools and prompt/output validation.

About the Role

Automation Quality Engineer responsible for designing, developing, and maintaining automation test frameworks in Java and/or Python to validate AI-enabled applications. The role focuses on testing LLM-based systems, RAG pipelines, vector databases, OpenAI APIs, and integrating AI-specific test strategies into CI/CD pipelines to ensure quality, performance, and reliability.

Job Description

Role

Seeking an Automation Quality Engineer with strong hands-on experience in Java and/or Python automation frameworks and practical exposure to AI-driven testing. The role focuses on validating intelligent applications and ensuring quality, reliability, performance, and security of AI-enabled systems.

Key Responsibilities

• Design, develop, and maintain automation test frameworks using Java and/or Python.
• Implement UI, API, and backend automation using tools such as Selenium, Playwright, REST Assured, PyTest, or similar.
• Validate LLM-based applications for prompt accuracy, response quality, hallucination detection, and consistency.
• Test and evaluate RAG pipelines, including document ingestion, embeddings, retrieval accuracy, and response relevance.
• Work with vector databases (FAISS, Pinecone, Chroma, Weaviate) to validate indexing, retrieval, and performance.
• Integrate and test applications using OpenAI APIs and similar LLM providers.
• Leverage LangChain (or equivalent frameworks) to test chains, agents, tools, and workflows.
• Create automated test cases for AI workflows, model outputs, prompt versions, and inference pipelines.
• Implement data-driven and AI-assisted testing strategies and participate in CI/CD pipelines to enforce quality gates.
• Collaborate with Data Scientists, Developers, and Product teams to define quality metrics and ensure compliance with security, performance, and scalability standards.

Requirements

• Strong programming skills in Python (and/or Java).
• Hands-on experience with automation testing frameworks such as Selenium, Playwright, REST Assured, PyTest, etc.
• Practical experience with LLMs (OpenAI GPT, Azure OpenAI, or similar).
• Solid understanding and exposure to RAG architectures.
• Experience working with vector databases for embeddings and semantic search (FAISS, Pinecone, Chroma, Weaviate).
• Hands-on usage of OpenAI APIs (chat/completions, embeddings, etc.).
• Experience with LangChain for prompt chaining, memory, and tool usage.
• Strong knowledge of API testing, SQL, and data validation.
• Familiarity with CI/CD tools such as Jenkins, GitHub Actions, Azure DevOps.
• Good understanding of SDLC, STLC, and Agile methodologies.

Good-to-Have

• Certifications in LLMs and/or Python.
• Exposure to AI-assisted testing strategies and emerging AI QA tools.
• Experience with performance testing of AI-enabled systems.
• Knowledge of cloud-based AI platforms and orchestration frameworks.

Skills

Automation Testing Test Framework Design API Testing UI Testing Data Validation SQL CI/CD SDLC STLC Agile Collaboration Performance Testing Security and Scalability Compliance AI-assisted Testing Prompt/LLM Validation