Senior Software Engineer, Test

Providius; Hamilton; Presencial; CAD 120.000 - 170.000; Jornada completa

Descripción de la vacante

As a hands-on contributor, you will build end-to-end test strategies, establish robust regression suites, and drive adoption of testing practices across the engineering organization, ensuring products ship with higher confidence and quality.

Responsabilidades

• Design, write, and maintain automated test suites across unit, integration, API, end-to-end, and regression testing
• Build and improve test infrastructure including fixtures, mocks, reusable test libraries, test data management, and simulation environments
• Build and maintain CI/CD test gates, smoke tests, regression suites, coverage reporting, and automated release-readiness checks
• Keep automated testing fast, reliable, and useful enough that developers trust it and use it regularly
• Build API and contract testing across services and integration boundaries
• Establish performance, load, stress, and soak testing for high-throughput and real-time systems
• Investigate and debug difficult failures across application, service, database, infrastructure, and deployment boundaries
• Work with engineers to determine appropriate test coverage and validation approaches for new features and systems
• Review test code with the same engineering standards applied to production code
• Help developers improve the quality and testability of their own code rather than making testing the responsibility of a separate QA silo
• Build reusable patterns and tooling that make it easier for engineers to write good tests
• Establish practical testing standards and help drive their adoption across engineering
• Identify gaps in existing test coverage and determine where deeper or different forms of validation are required
• Help adapt our testing practices to the increased pace and changing failure modes introduced by AI‑assisted software development
• Use modern development and AI tooling to improve the speed and effectiveness of test development while maintaining confidence in the results
• Validate that automated tests actually detect incorrect behaviour rather than simply confirming the current implementation
• Share testing practices, tooling, and lessons across the engineering team
• Build and maintain evaluation approaches for AI, ML, and LLM‑based product features
• Test non‑deterministic systems using representative datasets, repeated evaluation, appropriate metrics, and acceptance thresholds
• Establish regression testing for model, prompt, retrieval, and AI‑system changes
• Test AI integrations end to end, including tool use, schemas, authentication, authorization, error handling, and unexpected responses
• Help determine where automated evaluation is sufficient and where human review remains necessary
• Validate data and pipelines used to support AI and ML functionality
• Incorporate production failures and edge cases back into automated test and evaluation suites
• Help define evidence‑based release criteria for AI‑enabled product functionality

Descripción del empleo

Providius is looking for an experienced Senior Software Engineer, Test to help us build stronger, faster, and more reliable ways of validating complex software systems. This is a hands‑on engineering role for someone who enjoys writing code, building automation, debugging difficult problems, and improving how software gets tested across an engineering organization.

Position Overview

This is a hands‑on, individual‑contributor role. You will spend most of your week writing test code, building test infrastructure, and working directly with engineers, not writing test plans for others to execute. We are looking for someone to take our test automation and quality engineering practices to the next level. You will build the automated test suites and tooling underneath them, raise the standard for how the engineering team tests its own work, and help ensure our quality practices keep pace with how quickly modern software is now developed. Software gets written and changed faster than it did even a few years ago. AI‑assisted development accelerates that further, but it also changes where defects come from and what traditional reviews and test suites may miss. We want someone who sees that as an engineering problem: building the coverage, tooling, feedback loops, and development practices that allow the team to move quickly without sacrificing reliability. Our products also increasingly include AI and machine‑learning functionality, including LLM‑based assistants and integrations with AI clients. These systems introduce a different class of testing problem. Outputs may vary between runs, model or prompt changes can alter behaviour without traditional code changes, and quality may need to be measured across a set of results rather than with a simple pass or fail. You do not need to be an ML researcher, but you should have experience testing software with AI, ML, LLM, or other non‑deterministic components and understand how to build credible evidence that these systems are ready to ship. You will have meaningful autonomy and responsibility. You will work with engineering leadership to identify our biggest quality risks, establish practical approaches to addressing them, and implement the automation and tooling needed to make those approaches part of everyday development.

Duties and Responsibilities

• Design, write, and maintain automated test suites across unit, integration, API, end‑to‑end, and regression testing
• Build and improve test infrastructure including fixtures, mocks, reusable test libraries, test data management, and simulation environments
• Build and maintain CI/CD test gates, smoke tests, regression suites, coverage reporting, and automated release‑readiness checks
• Keep automated testing fast, reliable, and useful enough that developers trust it and use it regularly
• Build API and contract testing across services and integration boundaries
• Establish performance, load, stress, and soak testing for high‑throughput and real‑time systems
• Investigate and debug difficult failures across application, service, database, infrastructure, and deployment boundaries
• Work with engineers to determine appropriate test coverage and validation approaches for new features and systems
• Review test code with the same engineering standards applied to production code
• Help developers improve the quality and testability of their own code rather than making testing the responsibility of a separate QA silo
• Build reusable patterns and tooling that make it easier for engineers to write good tests
• Establish practical testing standards and help drive their adoption across engineering
• Identify gaps in existing test coverage and determine where deeper or different forms of validation are required
• Help adapt our testing practices to the increased pace and changing failure modes introduced by AI‑assisted software development
• Use modern development and AI tooling to improve the speed and effectiveness of test development while maintaining confidence in the results
• Validate that automated tests actually detect incorrect behaviour rather than simply confirming the current implementation
• Share testing practices, tooling, and lessons across the engineering team

AI and ML Quality

• Build and maintain evaluation approaches for AI, ML, and LLM‑based product features
• Test non‑deterministic systems using representative datasets, repeated evaluation, appropriate metrics, and acceptance thresholds
• Establish regression testing for model, prompt, retrieval, and AI‐system changes
• Test AI integrations end to end, including tool use, schemas, authentication, authorization, error handling, and unexpected responses
• Help determine where automated evaluation is sufficient and where human review remains necessary
• Validate data and pipelines used to support AI and ML functionality
• Incorporate production failures and edge cases back into automated test and evaluation suites
• Help define evidence‑based release criteria for AI‑enabled product functionality

Tech Stack

Our en