Remote AI Automation Test Engineer - LLM & Conversational AI

Sensiple; United States; Remote; USD 8.300 - 15.000; Vollzeit

Zusammenfassung

Sensiple is seeking an Automation Test Engineer with AI to join a remote Chennai-based team handling LLM and conversational AI testing, including prompt validation and non-deterministic outputs. The role involves designing benchmarks, performing functional and regression testing, and supporting automation for benchmarks across developer and QA cycles.

Candidates should have 3+ years in QA, bash/python skills, and familiarity with Playwright or Selenium, plus JMeter for load testing.

Qualifikationen

• 3+ years of experience in software/product quality assurance.
• Confident usage of Bash and Python.
• Experience with manual testing and test case execution.
• Basic familiarity with automated testing frameworks such as Playwright or Selenium.
• Experience with testing web protocols (WebSocket message payload) GEN AI / Agentic AI Testing.
• Basic knowledge of load and performance testing (e.g., JMeter).
• Strong analytical and problem-solving abilities.
• Systems thinking.
• Familiarity with prompt evaluation, function-calling, and tool-use testing.

Aufgaben

• Agentic Benchmark Design: Generate, import, create, extend, and manage benchmarks for non-deterministic LLM agents, validating behaviour and responses.
• Functional & Regression Testing: Grow benchmarks and create new ones as new functionality or issues are uncovered.
• Benchmarking Automation Support: Contribute to streamlining the benchmarking process in alignment with product development work.

Kenntnisse

Bash

Python

Manual testing

Playwright

Selenium

JMeter

Test automation

Analytical thinking

Tools

Playwright

Selenium

JMeter

Jobbeschreibung

Sensiple is seeking an Automation Test Engineer with AI to join a remote Chennai-based team handling LLM and conversational AI testing, including prompt validation and non-deterministic outputs. The role involves designing benchmarks, performing functional and regression testing, and supporting automation for benchmarks across developer and QA cycles.

Candidates should have 3+ years in QA, bash/python skills, and familiarity with Playwright or Selenium, plus JMeter for load testing.