Remote AI Agentic Tester: LLM & RAG Validation

Delan Associates, Inc; Denver (MO); A distancia; USD 70.000 - 110.000; Jornada completa

Descripción de la vacante

Delan Associates, Inc. is seeking an AI Agentic Tester in Denver, MO (Remote) to validate AI agent workflows and end-to-end AI applications. The role emphasizes testing of LLM responses, tool calls, and RAG pipelines, with an emphasis on accuracy and hallucination metrics.

The ideal candidate will develop automation scripts in Python and leverage Postman, Swagger, Selenium, Playwright, or PyTest to ensure high-quality AI solutions across Azure OpenAI, AWS Bedrock, and Gemini platforms.

Formación

• Must-Have Skills listed in the job ad.
• Experience validating AI agent workflows and end-to-end AI applications.
• Proficiency with LLM evaluation, accuracy, hallucination, and relevance metrics.

Responsabilidades

• Perform functional testing of AI agent workflows and end-to-end AI applications.
• Validate LLM responses for accuracy, relevance, consistency, and hallucination rates.
• Test RAG pipelines, prompt engineering, and AI agent orchestration.
• Execute API testing using Postman, Swagger, and REST APIs.
• Develop and maintain automation scripts using Python, Selenium, Playwright, or PyTest.
• Validate multi-agent interactions, tool calling, and workflow execution.

Conocimientos

AI Agentic Testing

Functional Testing

Generative AI Testing

LLMs

AI Agents & Agentic AI

RAG

Prompt Engineering Validation

AI Model Validation & Evaluation

API Testing

Python

Test Automation

AI Evaluation Metrics

End-to-End Workflow Testing

Multi-Agent Testing

Azure OpenAI

AWS Bedrock

Gemini

Herramientas

Postman

Swagger

REST APIs

Selenium

Playwright

PyTest

Python

Descripción del empleo

Delan Associates, Inc. is seeking an AI Agentic Tester in Denver, MO (Remote) to validate AI agent workflows and end-to-end AI applications. The role emphasizes testing of LLM responses, tool calls, and RAG pipelines, with an emphasis on accuracy and hallucination metrics.

The ideal candidate will develop automation scripts in Python and leverage Postman, Swagger, Selenium, Playwright, or PyTest to ensure high-quality AI solutions across Azure OpenAI, AWS Bedrock, and Gemini platforms.