AI Tester

Iris Software Inc.; Toronto; Hybrid; CAD 70,000 - 110,000; Full time

Job summary

Iris Software Inc. is seeking an AI Tester in Toronto, on a hybrid schedule (4 days in office, 1 day remote) for a long-term contract with a banking client. The role focuses on QA automation, Python-based tests, and hands-on GenAI/LLM testing with experience in eval tools and API validation.

The successful candidate will work within Iris’s engineering teams to ensure robust AI features pass through the CI/CD pipeline and meet banking-grade quality standards.

Qualifications

• 3+ years in software testing or QA automation
• Working Python skills required
• Hands-on experience testing LLM or GenAI applications
• Strong grasp of LLM concepts: prompts, embeddings, context windows, temperature, RAG
• Experience with eval tools such as DeepEval, Ragas, Promptfoo, LangSmith, or TruLens
• Comfort with API testing and working inside an existing CI/CD setup

Responsibilities

• Lead testing activities for AI/GenAI features in the banking client environment.
• Collaborate with devs to develop test strategies and plans.
• Implement and maintain test suites across APIs and integrations.
• Validate LLM prompts, embeddings, and context handling.
• Participate in CI/CD pipeline validation and regression testing.

Skills

Python

LLM/GenAI testing

API testing

CI/CD

QA automation

Tools

DeepEval

Ragas

Promptfoo

LangSmith

TruLens

Job description

Iris's Fortune 100 direct client is looking for AI Tester

Employment Type: Long term Contract

Hybrid: 4 days office , 1 day work from home

Location: Toronto, ON

Client: Banking

Key Responsibilities

• 3+ years in software testing or QA automation, with working Python skills


• Hands-on experience testing LLM or GenAI applications


• Strong grasp of LLM concepts: prompts, embeddings, context windows, temperature, RAG


• Experience with eval tools such as DeepEval, Ragas, Promptfoo, LangSmith, or TruLens


• Comfort with API testing and working inside an existing CI/CD setup

About Iris Software Inc.

With 4,000+ associates and offices in India, U.S.A. and Canada. Iris Software delivers technology services and solutions that help clients complete fast, far-reaching digital transformations and achieve their business goals. A strategic partner to Fortune 500 and other top companies in financial services and many other industries, Iris provides a value-driven approach - a unique blend of highly-skilled specialists, software engineering expertise, cutting-edge technology, and flexible engagement models. High customer satisfaction has translated into long-standing relationships and preferred-partner status with many of our clients, who rely on our 30+ years of technical and domain expertise to future-proof their enterprises. Associates of Iris work on mission-critical applications supported by a workplace culture that has won numerous awards in the last few years, including Certified Great Place to Work in India; Top 25 GPW in IT & IT-BPM; Ambition Box Best Place to Work, #3 in IT/ITES; and Top Workplace NJ-USA.