Senior ML/AI SDET: Quality, Automation & CI/CD

Sony Interactive Entertainment

Southwestern Ontario

On-site

CAD 115,000 - 172,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Sony Interactive Entertainment in Canada is seeking a Sr. Software Development Engineer in Test (ML/AI) to lead quality initiatives for ML-powered products and services.

You will define test strategies, develop scalable automation, and guide cross-functional teams across ML, engineering, and product groups. You’ll design scenario-based test suites, validate ML outputs, and drive release readiness while collaborating to embed testing in CI/CD pipelines and modernization efforts.

Qualifications

  • Bachelor’s degree in Computer Science or equivalent.
  • 5+ years of experience as an SDET or QE engineer focused on backend and distributed systems.
  • Experience using LLMs to generate, transform, and prioritize test cases for AI-powered experiences.
  • Experience with AI evaluation tooling, prompt evaluation frameworks, model monitoring, or human-in-the-loop review workflows.
  • Strong experience testing RESTful APIs, microservices, and distributed architectures.
  • Proficiency in Python, Java, JS or similar languages for automation development.
  • Hands-on experience with automation frameworks such as pytest, JUnit, Selenium, Playwright, Cypress, or Appium.
  • Experience with CI/CD systems and test pipelines (Jenkins, GitHub Actions, etc.).
  • Experience with cloud and container technologies (AWS, GCP, Kubernetes, Docker).
  • Familiarity with databases, monitoring, and observability tools.
  • Strong understanding of SDLC, Agile methodologies, and release processes.
  • Excellent problem-solving, debugging, and communication skills.

Responsibilities

  • Define and complete quality strategies, test plans, and automation coverage for ML-powered services and platform components.
  • Use LLMs and other AI-assisted techniques to generate, expand, and maintain high-value test cases for ML-powered workflows.
  • Design scenario-based test suites for AI features, including adversarial prompts, edge cases, ambiguous inputs, and underrepresented user scenarios.
  • Lead QE efforts for multi-functional projects, driving risk assessment, dependency management, and release readiness.
  • Design, develop, and maintain scalable automation frameworks for backend services, APIs, and ML inference systems using Python and/or Java.
  • Build automated validation for ML and LLM outputs, including ranking behavior, score distributions, prompt/response quality, hallucination indicators, and probabilistic model evaluation.
  • Debug test failures, service anomalies, model inconsistencies, and AI behavior regressions to identify root causes and drive resolution.
  • Perform functional, integration, regression, API, end-to-end, performance, and reliability testing for distributed systems.
  • Improve automation reliability, reduce flakiness, and optimize execution efficiency.
  • Partner with engineering and ML teams to integrate automated testing into CI/CD pipelines and release workflows.
  • Collaborate across teams to establish scalable quality standards, tooling, and guidelines.

Skills

SDET experience
API testing
CI/CD
Python
Java
LLM testing
Test automation
Debugging
Distributed systems
RESTful APIs
Jenkins
GitHub Actions
Kubernetes
Docker

Education

Bachelor’s degree in Computer Science or equivalent

Tools

pytest
JUnit
Selenium
Playwright
Cypress
Appium
Jenkins
GitHub Actions
AWS
GCP
Kubernetes
Docker

Job description

Sony Interactive Entertainment in Canada is seeking a Sr. Software Development Engineer in Test (ML/AI) to lead quality initiatives for ML-powered products and services.

You will define test strategies, develop scalable automation, and guide cross-functional teams across ML, engineering, and product groups. You’ll design scenario-based test suites, validate ML outputs, and drive release readiness while collaborating to embed testing in CI/CD pipelines and modernization efforts.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior ML/AI SDET: Quality & Automation Lead
Senior ML/AI SDET: Quality & Automation Lead

Sony Interactive Entertainment Inc. • Southwestern Ontario

On-site
CAD 115,000 - 172,000
Senior ML SDET: Test Automation & Validation
Senior ML SDET: Test Automation & Validation

Sony • Southwestern Ontario

Hybrid
CAD 114,000 - 143,000
Senior ML SDET: Scale Test Automation for AI Services
Senior ML SDET: Scale Test Automation for AI Services

Sony Interactive Entertainment • Southwestern Ontario

On-site
CAD 114,000 - 143,000
Senior ML SDET - Automation & Model Reliability
Senior ML SDET - Automation & Model Reliability

sonyinteractiveentertainmentglobal • Waterloo

Hybrid
CAD 100,000 - 140,000
Senior Software Development Engineer in Test, ML
Senior Software Development Engineer in Test, ML

Sony • Southwestern Ontario

Hybrid
CAD 114,000 - 143,000
Sr. Software Development Engineer in Test, AI/ML
Sr. Software Development Engineer in Test, AI/ML

playstation • Southwestern Ontario

Hybrid
CAD 115,000 - 172,000
Medical benefits
Dental benefits
Vision benefits
+4
Sr. Software Development Engineer in Test, AI/ML
Sr. Software Development Engineer in Test, AI/ML

Sony Interactive Entertainment • Southwestern Ontario

On-site
CAD 115,000 - 172,000
Sr. Software Development Engineer in Test, AI/ML
Sr. Software Development Engineer in Test, AI/ML

Sony Interactive Entertainment Inc. • Southwestern Ontario

On-site
CAD 115,000 - 172,000
Copy of Senior Software Development Engineer in Test, ML
Copy of Senior Software Development Engineer in Test, ML

sonyinteractiveentertainmentglobal • Waterloo

Hybrid
CAD 100,000 - 140,000
Sr. Software Development Engineer in Test, AI/ML
Sr. Software Development Engineer in Test, AI/ML

Sony Interactive Entertainment Inc. (Japan) • Southwestern Ontario

On-site
CAD 115,000 - 172,000
Medical, dental, vision coverage
401(k) matching
Paid time off
+2