AI Test Engineer: Validate AI & LLM Apps with Automation

Ethereum Technologies

Chicago (IL)

On-site

USD 90,000 - 120,000

Full time

8 hours ago

Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Ethereum Technologies is seeking an experienced AI Tester to validate AI-powered applications, ensuring accuracy, reliability, and performance across platforms. You will perform manual and automated testing, API testing, and prompt engineering, collaborating with developers and data scientists to improve quality.

The role emphasizes testing chatbots and GenAI workflows with focus on edge cases and hallucinations.

Qualifications

Strong experience in Manual and Automation Testing.
Knowledge of Python and API Testing.
Experience with AI/ML-based applications or LLM testing.
Understanding of ChatGPT, Generative AI, and Prompt Engineering.
Knowledge of Selenium, PyTest, or similar automation frameworks.
Experience with Postman, REST APIs, and SQL.
Strong analytical and problem-solving skills.

Responsibilities

Test AI-powered applications, chatbots, and LLM-based solutions.
Validate AI-generated responses for accuracy, relevance, and consistency.
Perform functional, regression, integration, and API testing.
Identify and report bugs, defects, and unexpected AI behavior.
Test different prompts and validate AI responses using prompt engineering techniques.
Work with developers and data science teams to improve application quality.
Perform automation testing using Python, Selenium, or similar tools.
Validate AI model outputs and test edge cases, hallucinations, and incorrect responses.