Senior AI/ML Quality Assurance Engineer

CDM Smith; Toronto; On-site; CAD 144,000 - 253,000; Full time

Job summary

Trinnex, a CDM Smith subsidiary, seeks a Senior AI/ML Quality Assurance Engineer to validate, test, and monitor production AI/ML solutions. You will ensure models are accurate, explainable, and compliant while partnering with Data Science, MLOps, and Governance teams.

You will lead automated testing, model validation, and documentation efforts across infrastructure, environmental compliance, and operational monitoring domains.

Qualifications

• Bachelor's degree required.
• 5 years of related experience.

Responsibilities

• Design, develop, and maintain automated testing frameworks for ML models, data pipelines, and analytics solutions.
• Validate various model types (tabular, geospatial, classification, risk-scoring) for accuracy and regulatory alignment.
• Perform model explainability and quality assessments with XAI techniques and document results for stakeholders.
• Collaborate with MLOps to implement monitoring for drift and degradation in production.
• Support investigations, rollback validation, and recovery testing for AI/ML systems.
• Evaluate Generative AI and RAG solutions for performance, safety, and guardrails.
• Develop validation reports and compliance artifacts with Governance teams.
• Conduct quality reviews of AI deliverables prior to client release.

Skills

Python

SQL

Git

Geospatial data tools

AI/ML testing

Model monitoring

Documentation

Communication

Cloud (GCP)

Education

Bachelor's degree

Tools

Docker

Kubernetes

GCP

MLflow

Deepchecks

Evidently AI

Grafana

Datadog

Job description

Company Overview

If you are passionate about water and technology, Trinnex is the place for you! Trinnex is a visionary company that is transforming the way water resources are managed and protected. By combining cutting-edge digital technologies, such as sensor/IoT data, models, geospatial data, and AI/machine learning, we create innovative, smart, and scalable solutions that make a difference. Whether it's optimizing water supply and demand, detecting leaks and anomalies, or enhancing water quality and resilience, Trinnex delivers value and impact to public sector clients across the country.

Job Description

At this time, we are only able to consider candidates who reside and work in British Columbia, Alberta, Manitoba, or Ontario, where the company is registered to operate.

Trinnex, wholly owned subsidiary of CDM Smith, is seeking a highly skilled and collaborative Senior AI/ML Quality Assurance Engineer to lead the validation, testing, and monitoring of production AI and machine learning solutions. This role is responsible for ensuring our predictive models are accurate, reliable, explainable, and compliant, while partnering with Data Science, MLOps, Consulting, and Governance teams to support responsible AI delivery. The ideal candidate combines expertise in AI/ML testing, model performance monitoring, and automation with the ability to translate complex technical findings into clear, actionable insights for both technical and non-technical stakeholders. This position will play a critical role in maintaining the quality and trustworthiness of Trinnex's AI-powered products across infrastructure, environmental compliance, and operational monitoring solutions.

Key Responsibilities

• Design, develop, and maintain automated testing frameworks to validate machine learning models, data pipelines, and predictive analytics solutions.
• Validate tabular, geospatial, classification, and risk-scoring models to ensure accuracy, reliability, and alignment with business and regulatory requirements.
• Perform model explainability and quality assessments using industry-standard XAI techniques and document results for internal and client-facing stakeholders.
• Partner with MLOps teams to implement monitoring, alerting, and performance tracking for data drift, concept drift, and model degradation in production environments.
• Support root cause analysis, incident investigations, rollback validation, and recovery testing for AI/ML systems.
• Assist in evaluating emerging Generative AI and Retrieval-Augmented Generation (RAG) solutions, including model performance, retrieval accuracy, safety, and guardrail effectiveness.
• Collaborate with Consulting and Governance teams to develop validation reports, model documentation, and compliance artifacts that support transparent and responsible AI practices.
• Conduct quality reviews of AI and data science deliverables prior to client release, identifying risks, data quality concerns, and model limitations.

Skills & Abilities

• Experience validating, testing, or monitoring production machine learning models, particularly classification, regression, or predictive analytics solutions.
• Strong proficiency in Python, SQL, and Git, with hands-on experience handling spatial data structures (e.g., GeoPandas, Shapely, or PostGIS).
• Experience with AI/ML testing, validation, and monitoring frameworks, including automated testing, model performance monitoring, and observability solutions. (e.g. pytest, MLflow, or Deepchecks; Evidently AI, Arize, or cloud observability stacks like GCP Production Monitoring/Cloud Monitoring, Grafana, or Datadog)
• Familiarity with Docker, Kubernetes, and cloud environments (GCP), including application monitoring and production support workflows.
• Ability to translate complex data science and model outputs into clear, actionable documentation for technical and business stakeholders.
• Understanding of model performance measurement, drift detection, and production monitoring concepts.
• Strong technical writing skills with experience creating and reviewing model documentation, validation reports, test plans, runbooks, APIs, or other technical artifacts.
• Ability to collaborate across multidisciplinary teams, including Data Science, MLOps, Software Engineering, Product Management, and Consulting organizations.

Qualifications

• Bachelor's degree.
• 5 years of related experience.

Equivalent additional directly related experience will be considered in lieu of a college degree. Domestic and/or international travel may be required. The frequency of travel is contingent on specific duties, responsibilities, and the essential functions of the position, which may vary depending on workload and project demands.

Preferred Qualifications

• Bachelor's degree in Computer Science, Software Engineering, Data Science, Geographic Information Systems (GIS), Engineering, or a related quantitative field.
• Experience validating machine learning models, datasets, or analytical solutions within infrastructure, utilities, environmental compliance, water, or wastewater domains.
• Familiarity with geospatial data, spatial analytics, or GIS-based applications.
• Experience working with time-series, sensor, IoT, or SCADA data in operational environments.
• Knowledge of AI governance, model risk management, or responsible AI frameworks.
• Familiarity with cloud-based AI/ML platforms and production monitoring environments.
• Experience evaluating or testing Generative AI, large language model (LLM), or Retrieval-Augmented Generation (RAG) solutions.
• Understanding model explainability, fairness, bias assessment, and AI transparency concepts.

Amount Of Travel Required

No Travel is required

Additional Information

This job posting is for an existing vacancy.

Background Check and Drug Testing Information

CDM Smith Inc. and its divisions and subsidiaries (hereafter collectively referred to as “CDM Smith”) reserves the right to require background checks including criminal, employment, education, licensure, etc. as well as credit and motor vehicle when applicable for certain positions. In addition, CDM Smith may conduct drug testing for designated positions. Background checks are conducted after an offer of employment has been made in the United States. The timing of when background checks will be conducted on candidates for positions outside the United States will vary based on country statutory law but in no case, will the background check precede an interview. CDM Smith will conduct interviews of qualified individuals prior to requesting a criminal background check, and no job application submitted prior to such interview shall inquire into an applicant's criminal history. If this position is subject to a background check for any convictions related to its responsibilities and requirements, employment will be contingent upon successful completion of a background investigation including criminal history. Criminal history will not automatically disqualify a candidate. In addition, during employment individuals may be required by CDM Smith or a CDM Smith client to successfully complete additional background checks, including motor vehicle record as well as drug testing.

Pay Range Min

CAD $144,477.00

Pay Range Max

CAD $252,824.00

Additional Pay Range Information

CDM Smith is committed to fair and equitable compensation practices. This pay range is a good faith estimate representative of all experience levels for the geographic location assigned to the position. In addition to geographic location, a candidate’s salary is determined by several other factors including, but not limited to, the role, function and associated responsibilities, relevant work experience, skills, required certifications, and education/training.

Additional Compensation

All bonuses at CDM Smith are discretionary and may or may not apply to this position.

Visa Sponsorship Available

No-Please note that all applicants must be legally eligible to work in Canada, for the Company, at the time of hire. Furthermore, this is not a position for which the Company is offering immigration application sponsorship or support

Accessibility Request for Applicants to Canadian Positions

CDM Smith attracts the best people in the industry, supporting their efforts to learn and grow. We strive to create a challenging and progressive work environment. We provide career opportunities that span a variety of disciplines and geographic locations, with projects that our employees plan, design, build and operate as diverse as the needs of our clients. CDM Smith is an Equal Opportunity/Affirmative Action employer. Given our ongoing commitment to fostering an inclusive workplace, CDM Smith will consider all other accommodation requests related to protected characteristics in good faith, in accordance with its internal policies and procedures and all applicable laws.

If you require a reasonable accommodation for any part of the application process, please contact our Human Resources Department at +1.617.452.6000 and ask to speak with the Employee Relations Manager. Please have the requisition number of the position you are applying to available during the call.

Agency Disclaimer

All vendors must have a signed CDM Smith Placement Agreement from the CDM Smith Talent Acquisition Manager to receive payment for your placement. Verbal or written commitments from any other member of the CDM Smith Inc. and its divisions and subsidiaries staff will not be considered binding terms. All unsolicited resumes sent to CDM Smith Inc. and its divisions and subsidiaries and any resume submitted to any employee outside of CDM Smith Talent Acquisition Team will be considered property of CDM Smith. CDM Smith Inc. and its divisions and subsidiaries will not be held liable to pay a placement fee.