Senior SDET, ML/AI QA & Automation Lead

PlayStation; San Mateo (CA); Hybrid; USD 183,000 - 275,000; Full time

Benefits offered by this job

Medical Insurance

Dental Insurance

Vision Insurance

401(k) Matching

Paid Time Off

Wellness Program

Employee Discounts

Job summary

Sony Interactive Entertainment seeks a Senior Software Development Engineer In Test with ML/AI focus to lead quality initiatives on ML-powered services and model evaluation. You will design automation frameworks, guide QA strategy, and collaborate across ML, engineering, product, and infrastructure teams to ensure reliable releases.

You will build scalable test suites, validate ML/LLM outputs, and drive risk assessment and release readiness across complex projects.

Qualifications

• Bachelor’s degree in Computer Science or equivalent
• 5+ years of experience as an SDET or QE engineer focused on backend and distributed systems.
• Experience using LLMs to generate, transform, and prioritize test cases for AI-powered experiences.
• Experience with AI evaluation tooling, prompt evaluation frameworks, model monitoring, or human-in-the-loop review workflows.
• Strong experience testing RESTful APIs, microservices, and distributed architectures.
• Proficiency in Python, Java, JS or similar languages for automation development.
• Hands‑on experience with automation frameworks such as pytest, JUnit, Selenium, Playwright, Cypress, or Appium.
• Experience with CI/CD systems and test pipelines (Jenkins, GitHub Actions, etc.).
• Experience with cloud and container technologies (AWS, GCP, Kubernetes, Docker).
• Familiarity with databases, monitoring, and observability tools.
• Strong understanding of SDLC, Agile methodologies, and release processes.
• Excellent problem‑solving, debugging, and communication skills.

Responsibilities

• Define and complete quality strategies, test plans, and automation coverage for ML-powered services and platform components.
• Use LLMs and other AI-assisted techniques to generate, expand, and maintain high-value test cases for ML-powered workflows.
• Design scenario-based test suites for AI features, including adversarial prompts, edge cases, ambiguous inputs, and underrepresented user scenarios.
• Lead QE efforts for multi-functional projects, driving risk assessment, dependency management, and release readiness.
• Design, develop, and maintain scalable automation frameworks for backend services, APIs, and ML inference systems using Python and/or Java.
• Build automated validation for ML and LLM outputs, including ranking behavior, score distributions, prompt/response quality, hallucination indicators, and probabilistic model evaluation.
• Debug test failures, service anomalies, model inconsistencies, and AI behavior regressions to identify root causes and drive resolution.
• Perform functional, integration, regression, API, end-to-end, performance, and reliability testing for distributed systems.
• Improve automation reliability, reduce flakiness, and optimize execution efficiency.
• Partner with engineering and ML teams to integrate automated testing into CI/CD pipelines and release workflows.
• Collaborate across teams to establish scalable quality standards, tooling, and guidelines.

Skills

SDET experience

Backend testing

Distributed systems

LLMs

AI evaluation

Test automation

Python

Java

REST APIs

CI/CD

Cloud platforms

Docker

Kubernetes

Jenkins

GitHub Actions

Monitoring tools

Education

Bachelor's degree in Computer Science or equivalent

Tools

pytest

JUnit

Selenium

Playwright

Cypress

Appium

Job description

Sony Interactive Entertainment seeks a Senior Software Development Engineer In Test with ML/AI focus to lead quality initiatives on ML-powered services and model evaluation. You will design automation frameworks, guide QA strategy, and collaborate across ML, engineering, product, and infrastructure teams to ensure reliable releases.

You will build scalable test suites, validate ML/LLM outputs, and drive risk assessment and release readiness across complex projects.