Senior ML/AI SDET: Quality & Automation Lead

Sony Interactive Entertainment Inc.; Southwestern Ontario; On-site; CAD 115,000 - 172,000; Full time

Job summary

Sony Interactive Entertainment is seeking a Sr. Software Development Engineer in Test, ML/AI to join our AI quality team in Canada. You will help shape testing strategy for ML-powered features, build automation, and lead quality initiatives across cross-functional teams.

You will collaborate with ML engineers, software engineers, and product teams to ensure robust test coverage, deterministic results, and reliable releases in highly scalable distributed systems.

Qualifications

• Bachelor's degree in Computer Science or equivalent.
• 5+ years as SDET/QE focused on backend and distributed systems.
• Experience using LLMs to generate, transform, and prioritize test cases for AI-powered experiences.
• Experience with AI evaluation tooling, prompt evaluation frameworks, model monitoring, or human-in-the-loop review workflows.
• Strong experience testing RESTful APIs, microservices, and distributed architectures.
• Proficiency in Python, Java, JS or similar languages for automation development.
• Hands-on experience with automation frameworks such as pytest, JUnit, Selenium, Playwright, Cypress, or Appium.
• Experience with CI/CD systems and test pipelines (Jenkins, GitHub Actions, etc.).
• Experience with cloud and container technologies (AWS, GCP, Kubernetes, Docker).
• Familiarity with databases, monitoring, and observability tools.
• Strong understanding of SDLC, Agile methodologies, and release processes.
• Excellent problem-solving, debugging, and communication skills.

Responsibilities

• Define and complete quality strategies, test plans, and automation coverage for ML-powered services and platform components.
• Use LLMs and other AI-assisted techniques to generate, expand, and maintain high-value test cases for ML-powered workflows.
• Design scenario-based test suites for AI features, including adversarial prompts, edge cases, ambiguous inputs, and underrepresented user scenarios.
• Lead QE efforts for multi-functional projects, driving risk assessment, dependency management, and release readiness.
• Design, develop, and maintain scalable automation frameworks for backend services, APIs, and ML inference systems using Python and/or Java.
• Build automated validation for ML and LLM outputs, including ranking behavior, score distributions, prompt/response quality, hallucination indicators, and probabilistic model evaluation.
• Debug test failures, service anomalies, model inconsistencies, and AI behavior regressions to identify root causes and drive resolution.
• Perform functional, integration, regression, API, end-to-end, performance, and reliability testing for distributed systems.
• Improve automation reliability, reduce flakiness, and optimize execution efficiency.
• Partner with engineering and ML teams to integrate automated testing into CI/CD pipelines and release workflows.
• Collaborate across teams to establish scalable quality standards, tooling, and guidelines.

Skills

Python

Java

LLMs for testing

RESTful APIs

Distributed systems

CI/CD tooling

JUnit/Pytest

Docker/Kubernetes

Cloud (AWS/GCP)

Automation frameworks

Education

Bachelor's in Computer Science

Tools

pytest

JUnit

Selenium

Playwright

Cypress

Appium

Job description

Sony Interactive Entertainment is seeking a Sr. Software Development Engineer in Test, ML/AI to join our AI quality team in Canada. You will help shape testing strategy for ML-powered features, build automation, and lead quality initiatives across cross-functional teams.

You will collaborate with ML engineers, software engineers, and product teams to ensure robust test coverage, deterministic results, and reliable releases in highly scalable distributed systems.