Senior AI/ML QA Engineer: Production Validation & Monitoring

CDM Smith; Toronto; On-site; CAD 144,000 - 253,000; Full time

Job summary

Trinnex, a CDM Smith subsidiary, seeks a Senior AI/ML Quality Assurance Engineer to validate, test, and monitor production AI/ML solutions. You will ensure models are accurate, explainable, and compliant while partnering with Data Science, MLOps, and Governance teams.

You will lead automated testing, model validation, and documentation efforts across infrastructure, environmental compliance, and operational monitoring domains.

Qualifications

• Bachelor's degree required.
• 5 years of related experience.

Responsibilities

• Design, develop, and maintain automated testing frameworks for ML models, data pipelines, and analytics solutions.
• Validate various model types (tabular, geospatial, classification, risk-scoring) for accuracy and regulatory alignment.
• Perform model explainability and quality assessments with XAI techniques and document results for stakeholders.
• Collaborate with MLOps to implement monitoring for drift and degradation in production.
• Support investigations, rollback validation, and recovery testing for AI/ML systems.
• Evaluate Generative AI and RAG solutions for performance, safety, and guardrails.
• Develop validation reports and compliance artifacts with Governance teams.
• Conduct quality reviews of AI deliverables prior to client release.

Skills

Python

SQL

Git

Geospatial data tools

AI/ML testing

Model monitoring

Documentation

Communication

Cloud (GCP)

Education

Bachelor's degree

Tools

Docker

Kubernetes

GCP

MLflow

Deepchecks

Evidently AI

Grafana

Datadog

Job description

Trinnex, a CDM Smith subsidiary, seeks a Senior AI/ML Quality Assurance Engineer to validate, test, and monitor production AI/ML solutions. You will ensure models are accurate, explainable, and compliant while partnering with Data Science, MLOps, and Governance teams.

You will lead automated testing, model validation, and documentation efforts across infrastructure, environmental compliance, and operational monitoring domains.