Automation Tester for AI-Driven Applications

Sourceworx; Johannesburg; Hybrid; ZAR 800 000 - 1 000 000; Full time

Job summary

A technology firm specializing in AI is seeking a Test Engineer to validate the functionality and performance of AI-driven systems. The role requires expertise in test engineering, automation tools, and AI technologies like GPT-4. Candidates should have 5-7 years of experience and be detail-oriented problem solvers able to work in an agile environment. This position offers opportunities for remote or onsite work in Johannesburg.

Qualifications

• At least 5 - 7 years in test engineering.
• Familiar with Microsoft Responsible AI Toolbox and AI Foundry.
• Experience with automation frameworks.

Responsibilities

• Test and evaluate AI-driven applications.
• Report daily and weekly test results.
• Conduct defect tracking and root cause analysis.
• Participate in daily agile meetings.

Skills

Test engineering experience

Knowledge of GPT-4 and LLMs

Automation tools expertise

CI/CD integration proficiency

Quality Assurance principles

Experience in Fintech/BFSI domain

Agile methodologies understanding

Tools

Selenium

JMeter

LoadRunner

Appium

Jenkins

GitLab CI/CD

AzureDevOps

Jira

Job description

Job Purpose

The tester will be responsible for validating the functionality, performance, and reliability of AI-driven systems and applications. This includes designing and executing test cases for automation flows, and intelligent features integrated into enterprise platforms. This role is not about using AI for testing. Instead, this is about testing applications where AI has been used. Testers are required to understand the probabilistic nature of AI driven systems and should be able to test applications for factual correctness, hallucinations, guardrail breaches, and safety. Testers should have firm understanding of GenAI tools such as ChatGTP, Gemini, etc. with an understanding of the patterns on how they provide answers and where they can provide inconsistent and incorrect answers. Elementary courses in prompt engineering will be highly beneficial.

Mandatory Skills

• Experience: At least 5 - 7 years in the field of test engineering.
• AI Tools: Knowledge of GPT-4.0 or other LLMs, as well as expertise in prompt engineering, is highly beneficial.
• Familiarity with the Microsoft Responsible AI Toolbox, AI Foundry, and Guardrail.
• Automation Tools: Significant expertise in automation tools such as Selenium, JMeter, LoadRunner, or Appium for automating web, mobile, and performance testing.
• CI/CD Integration: A strong grasp on continuous integration and continuous deployment tools such as Jenkins, Bamboo, or GitLab CI/CD to automate build and deployment processes.
• Quality Assurance: Thorough understanding of Quality Assurance principles, testing methodologies, and best practices. The ability to develop robust automation frameworks to ensure the highest standards of quality in our products.
• Preferred Skills: Experience in the Fintech/BFSI domain; Availability for remote or onsite work; Proficiency in automation and framework; Expertise in validating data correction and hallucination; Expertise in AzureDevOps or Jira.
• Industry Standards: Understanding of software development methodologies like Agile, Scrum, and DevOps practices.

Roles and Responsibilities

• Role Focus: Testing and evaluating AI Driven Applications.
• Skills Required: Proficiency in testing and evaluating technology.
• Work Environment: Agile model, collaboration with Development and Integration teams.
• Independence: Ability to work independently and bring innovative ideas.
• Daily and weekly test reporting.
• Defect tracking.
• Attending daily stand-up and retrospective meetings.
• Problem-solving and root cause analysis.
• Automation scripts and test case accuracy.
• Analytical Skills: Ability to analyze test results, identify trends, and derive meaningful insights.
• Desired Traits: Detail-oriented, innovative, analytical.