AI Quality Engineer

Tekskills Inc.; Bengaluru; In loco; INR 1.200.000 - 1.800.000; Tempo pieno

Descrizione del lavoro

Tekskills Inc. in Pune is seeking an AI Quality Engineer (Tester) to lead testing and evaluation of AI applications, including LLMs, RAG, and agent workflows, ensuring quality, safety, and business alignment.

You will design and execute functional and regression test cases, evaluate AI systems with specialized tools, perform API and performance tests, and integrate automated tests into CI/CD pipelines while collaborating with cross-functional teams.

Competenze

• 3+ years in QA/testing (API + performance testing)
• Experience with automation tools (Selenium, Playwright)
• Programming knowledge (Python/TypeScript)
• CI/CD, version control, and cloud basics (AWS/Azure/GCP)

Mansioni

• Design & execute functional/regression test cases (SIT/UAT)
• Evaluate LLMs using frameworks (e.g., RAGAS, PromptFoo)
• Test RAG systems: retrieval accuracy, hallucination, grounding
• Validate agent workflows: multi-step logic, tool usage, structured outputs
• Perform API, performance, and load testing (latency, throughput, cost)
• Conduct security testing (prompt injection, data privacy, guardrails)
• Integrate automated tests into CI/CD pipelines
• Monitor, report, and optimize AI system performance
• Collaborate with cross-functional teams and support release readiness

Conoscenze

QA testing

Automation mindset

Programming (Python/TypeScript)

CI/CD basics

Cloud basics (AWS/Azure/GCP)

Strumenti

Selenium

Playwright

Postman

REST Assured

k6

JMeter

Descrizione del lavoro

Role: AI Quality Engineer (Tester)
Experience: 3-4 years
Location: Pune

Purpose:
Lead testing and evaluation of AI applications (LLMs, RAG, agent workflows) to ensure quality, safety, performance, and business alignment.

Key Responsibilities

• Design & execute functional/regression test cases (SIT/UAT)
• Evaluate LLMs using frameworks (e.g., RAGAS, PromptFoo)
• Test RAG systems: retrieval accuracy, hallucination, grounding
• Validate agent workflows: multi-step logic, tool usage, structured outputs
• Perform API, performance, and load testing (latency, throughput, cost)
• Conduct security testing (prompt injection, data privacy, guardrails)
• Integrate automated tests into CI/CD pipelines
• Monitor, report, and optimize AI system performance
• Collaborate with cross-functional teams and support release readiness

Required Skills

• 3+ years in QA/testing (API + performance testing)
• Experience with automation tools (Selenium, Playwright)
• Programming knowledge (Python/TypeScript)
• CI/CD, version control, and cloud basics (AWS/Azure/GCP)

Preferred

• AI/ML and MLOps understanding
• Experience with LLM evaluation tools (LangSmith, RAGAS, TruLens, etc.)
• Knowledge of AI security testing and observability tools

Tools

Playwright, Selenium, Postman, REST Assured, k6, JMeter, CI/CD tools

Traits

Strong communication, problem-solving, and quality-focused mindset