Remote Senior SDET – AI & Automation for Secure Apps

Openkrill; United States; Remote; USD 140,000 - 190,000; Full time

Benefits offered by this job

Medical Health Benefits

Job summary

Keeper Security is seeking a Senior SDET, AI & Automation to embed within the AI team and shape automated quality across AI-powered products. The role focuses on deterministic and non-deterministic AI validation, building scalable test frameworks and integrating quality gates into CI/CD.

You will work with AI engineers, software developers and product managers to define strategies, establish acceptance criteria, and ensure robust automated coverage across APIs, services and AI models in a

Qualifications

• 5+ years in SDET or code-intensive quality role
• Hands-on testing of AI/ML or model-integrated software
• Experience defining evaluation criteria for AI-driven functionality
• Strong programming skills in one or more listed languages
• Experience building automated test frameworks
• Experience integrating tests into CI/CD pipelines
• Familiarity with AI-assisted development tools

Responsibilities

• Partner with AI engineers, software engineers and product managers to define test strategy and automation
• Design and maintain automated testing frameworks for AI-powered applications and APIs
• Develop testing strategies for deterministic and non-deterministic AI outputs
• Define evaluation criteria, tolerances and quality thresholds for AI models
• Build automated regression, integration, API and end-to-end coverage
• Develop test harnesses for prompts, models and tool-calling workflows
• Validate AI behavior across models, prompts and configurations
• Integrate automated validation into CI/CD pipelines
• Create test datasets, mocks and synthetic data for AI validation
• Analyze failures to distinguish defects from AI variability
• Improve observability and testability to reduce flaky tests
• Track test cases and coverage with test management tools

Skills

Python

Java

JavaScript/TypeScript

Rust

AI/ML testing

REST APIs testing

Test automation

CI/CD pipelines

Deterministic and non-deterministic AI

Education

Bachelor’s degree in Computer Science or related field

Tools

Postman

REST Assured

GitHub Actions

Jenkins

GitLab CI

TestRail

Docker

Kubernetes

Job description

Keeper Security is seeking a Senior SDET, AI & Automation to embed within the AI team and shape automated quality across AI-powered products. The role focuses on deterministic and non-deterministic AI validation, building scalable test frameworks and integrating quality gates into CI/CD.

You will work with AI engineers, software developers and product managers to define strategies, establish acceptance criteria, and ensure robust automated coverage across APIs, services and AI models in a