AI Architect / Tester

49 Solutions; Ottawa; On-site; CAD 80,000 - 110,000; Full time

Job summary

A technology consulting firm is seeking an experienced AI Tester / Architect Consultant to join their client at IRCC. You will ensure the functionality and reliability of AI applications, develop test strategies, and collaborate with teams to enhance AI performance. Minimum 5 years in software testing and a valid government security clearance are required. This role supports remote/hybrid work.

Benefits

Remote/Hybrid Work

Qualifications

• Minimum 5 years of experience in software testing or test automation, specifically with AI applications.
• Strong proficiency with AI frameworks, testing tools, and AI technologies.
• Experience with AI agents, RAG, LLMs, and machine learning models.
• Strong programming skills in Python, Java, or R.
• Familiarity with DevOps processes and CI/CD pipelines.
• Excellent analytical, problem-solving, and communication skills.
• A valid Canadian Government Security Clearance is a non-negotiable.

Responsibilities

• Design, develop, and execute test strategies for AI applications.
• Leverage AI tools to test applications regularly.
• Evaluate and validate machine learning models, algorithms, and AI frameworks.
• Implement test automation for AI systems using Python, Java, or R.
• Collaborate with cross-functional teams to resolve performance or functionality issues.
• Support DevOps processes and CI/CD pipelines in AI deployment.

Skills

Software testing

Test automation

AI frameworks

Analytical skills

Problem-solving

Communication

Education

Degree in Computer Science, Engineering, or related field

Tools

TensorFlow

Keras

PyTorch

Python

Java

R

Job description

Overview

49 Solutions is seeking an experienced AI Tester / Architect Consultant to join our client at IRCC for innovative projects within the Canadian Federal Government. You will play a critical role in ensuring the functionality, performance, and reliability of AI applications, combining technical expertise with a deep understanding of AI technologies to contribute effectively across multiple initiatives.

Responsibilities

• Design, develop, and execute test strategies for AI applications, including AI agents, RAG, and LLMs. Create test plans and cases, conduct functional, performance, and integration testing, identify potential model biases and edge cases, and collaborate with developers and data scientists to improve AI reliability and accuracy.
• Leverage AI tools to test applications on a regular basis, using prompts and outputs to generate and maintain test cases that evolve as AI models are updated or modified.
• Evaluate and validate machine learning models, algorithms, and AI frameworks such as TensorFlow, Keras, and PyTorch to ensure optimal performance and accuracy.
• Implement test automation for AI systems using Python, Java, or R to streamline testing processes and ensure repeatable results.
• Collaborate with cross-functional teams to identify, analyze, and resolve performance or functionality issues, ensuring AI applications meet project requirements.
• Support DevOps processes and CI/CD pipelines in AI application deployment, enabling seamless integration and continuous delivery.
• Ensure compliance with best practices and security requirements for federal government projects, maintaining high standards of quality and reliability.

Qualifications

• Minimum 5 years of experience in software testing or test automation, specifically with AI applications.
• Strong proficiency with AI frameworks, testing tools, and AI technologies.
• Experience with AI agents, RAG, LLMs, and machine learning models.
• Strong programming skills in Python, Java, or R.
• Familiarity with DevOps processes and CI/CD pipelines.
• Excellent analytical, problem-solving, and communication skills.
• A degree in Computer Science, Engineering, or a related field.
• A valid Canadian Government Security Clearance is a non-negotiable.

Benefits

Remote/Hybrid Work