Software Development Engineer in Test, ML/AI

Sonyinteractiveentertainmentglobal; California (MO); Hybrid; USD 140,500 - 210,700; Full time

Benefits offered by this job

Medical, dental, and vision benefits

401(k) matching

Paid time off

Employee discounts for Sony products

Wellness program

Job summary

Sonyinteractiveentertainmentglobal in California is looking for a Software Development Engineer In Test specializing in ML/AI. You will lead quality initiatives and develop automation frameworks while collaborating with cross-functional teams. The role requires 3+ years in quality engineering, involving a deep understanding of AI technologies and automation tools.

The estimated base pay range is $140,500 to $210,700 USD, along with a comprehensive benefits package. Join us to drive innovation and improve quality in ML-powered services.

Qualifications

• 3+ years of experience as an SDET or QE engineer focused on backend and distributed systems.
• Experience using LLMs to generate, transform, and prioritize test cases.
• Strong experience testing microservices and distributed architectures.

Responsibilities

• Define and complete quality strategies, test plans, and automation coverage.
• Lead QE efforts for multi-functional projects.
• Design, develop, and maintain scalable automation frameworks.

Skills

Experience with AI evaluation tooling

Proficiency in Python

Experience testing RESTful APIs

Experience with automation frameworks like pytest or Selenium

Hands-on experience with CI/CD systems

Strong problem-solving skills

Education

Bachelor’s degree in Computer Science or equivalent

Tools

Jenkins

Docker

Kubernetes

Job description

Software Development Engineer In Test, ML/AI

Overview

We are hiring a Software Engineer in Test specializing in ML/AI quality, including automation for model evaluation, LLM‑assisted test generation, and validation of AI‑powered workflows. The role includes guiding quality strategy and developing automation frameworks. You will also manage implementation for complex, cross‑functional, machine‑learning powered products and services. This role involves more than test development. You will lead quality initiatives from start to finish, ensuring teams stay synchronized, dependencies are managed, and releases are delivered confidently. You will work closely with ML, engineering, product, and infrastructure teams. You will shape how quality is built, monitored, and expanded while leading embedded QE efforts within projects.

What you’ll do

• Define and complete quality strategies, test plans, and automation coverage for ML‑powered services and platform components.
• Use LLMs and other AI‑assisted techniques to generate, expand, and maintain high‑value test cases for ML‑powered workflows.
• Design scenario‑based test suites for AI features, including adversarial prompts, edge cases, ambiguous inputs, and under‑represented user scenarios.
• Lead QE efforts for multi‑functional projects, driving risk assessment, dependency management, and release readiness.
• Design, develop, and maintain scalable automation frameworks for backend services, APIs, and ML inference systems using Python and/or Java.
• Build automated validation for ML and LLM outputs, including ranking behavior, score distributions, prompt/response quality, hallucination indicators, and probabilistic model evaluation.
• Debug test failures, service anomalies, model inconsistencies, and AI behavior regressions to identify root causes and drive resolution.
• Perform functional, integration, regression, API, end‑to‑end, performance, and reliability testing for distributed systems.
• Improve automation reliability, reduce flakiness, and optimize execution efficiency.
• Partner with engineering and ML teams to integrate automated testing into CI/CD pipelines and release workflows.
• Collaborate across teams to establish scalable quality standards, tooling, and guidelines.

Required qualifications

• Bachelor’s degree in Computer Science or equivalent practical experience.
• 3+ years of experience as an SDET or QE engineer focused on backend and distributed systems.
• Experience using LLMs to generate, transform, and prioritize test cases for AI‑powered experiences.
• Experience with AI evaluation tooling, prompt evaluation frameworks, model monitoring, or human‑in‑the‑loop review workflows.
• Strong experience testing RESTful APIs, microservices, and distributed architectures.
• Proficiency in Python, Java, JS or similar languages for automation development.
• Hands‑on experience with automation frameworks such as pytest, JUnit, Selenium, Playwright, Cypress, or Appium.
• Experience with CI/CD systems and test pipelines (Jenkins, GitHub Actions, etc.).
• Experience with cloud and container technologies (AWS, GCP, Kubernetes, Docker).
• Familiarity with databases, monitoring, and observability tools.
• Strong understanding of SDLC, Agile methodologies, and release processes.
• Excellent problem‑solving, debugging, and communication skills.

Preferred qualifications

• Experience validating ML outputs using statistical analysis or scenario‑based testing approaches.
• Familiarity with ML infrastructure, data pipelines, or model‑serving platforms (Seldon, KServe, Ray Serve, etc.).
• Prior work in content moderation ML, security, fraud detection, or adversarial ML.
• Experience testing high‑scale, low‑latency online services.
• Experience with Databricks or similar ML platform tooling.
• Familiarity with Node.js, React, or modern frontend technologies.
• Experience testing mobile, console, or other non‑PC platforms.

What sets you apart

• Strong combination of automation engineering and delivery ownership.
• Ability to drive quality across complex cross‑functional initiatives.
• Practical understanding of how to test non‑deterministic AI systems and separate model variance from quality regressions.
• Proven risk management and dependency coordination skills.
• Ability to influence engineering teams and promote quality guidelines.
• Passion for scalable, reliable, and maintainable automation systems.

At SIE, we consider several factors when setting each role’s base pay range, including competitive benchmarking data for the market and geographic location. Please note that the base pay range may vary in line with our hybrid working policy and individual base pay will be determined based on job‑related factors which may include knowledge, skills, experience, and location. In addition, this role is eligible for SIE’s top‑tier benefits package that includes medical, dental, vision, matching 401(k), paid time off, wellness program and coveted employee discounts for Sony products. This role also may be eligible for a bonus package. Click here to learn more.

The estimated base pay range for this role is listed below.

$140,500 — $210,700 USD

Equal Opportunity Statement:

Sony is an Equal Opportunity Employer. All persons will receive consideration for employment without regard to gender (including gender identity, gender expression and gender reassignment), race (including colour, nationality, ethnic or national origin), religion or belief, marital or civil partnership status, disability, age, sexual orientation, pregnancy, maternity or parental status, trade union membership or membership in any other legally protected category. We strive to create an inclusive environment, empower employees and embrace diversity. We encourage everyone to respond. Sony Interactive Entertainment is a Fair Chance employer and qualified applicants with arrest and conviction records will be considered for employment.