Software Development Test Engineer (SDET), AI & Automation

keepersecurity; United States; Remote; USD 120,000 - 160,000; Full time

Benefits offered by this job

Remote position

Hybrid schedule in CA/IL

Medical, dental, and vision coverage

Employer-paid life insurance

Disability insurance

401(k) with Roth and Traditional

Generous paid time off

Bereavement and jury duty leave

Job summary

Keeper Security is seeking a Senior Software Development Engineer in Test (SDET) to partner with an AI engineering team on AI-powered cybersecurity products. You will shape quality engineering, blending traditional testing with AI/ML validation and building automated frameworks.

This fully remote role offers a hybrid schedule for candidates in the El Dorado Hills, CA or Chicago, IL metro areas and requires 5+ years in quality engineering and strong coding skills.

Qualifications

• 5+ years in SDET, test automation, or related quality engineering role.
• Hands-on experience testing AI/ML, LLM-powered, or model-integrated software.
• Strong programming skills in Python, Java, JavaScript/TypeScript, or Rust.
• Experience designing, building, and maintaining automated test frameworks.
• Experience integrating automated tests into CI/CD pipelines (GitHub Actions, Jenkins, GitLab CI).

Responsibilities

• Collaborate with AI engineers, software engineers, and product managers to define test strategy, acceptance criteria, and automation requirements.
• Design and maintain automated test frameworks for AI-powered applications, backend services, and APIs.
• Develop evaluation strategies for deterministic and non-deterministic AI/ML outputs.
• Define measurable quality thresholds and evaluation criteria for AI/LLM functionality.
• Build regression, integration, API, and end-to-end coverage alongside feature development.
• Integrate automated validation and quality gates into CI/CD pipelines.

Skills

Python

Java

JavaScript/TypeScript

Rust

Automated test frameworks

CI/CD integration

Tools

GitHub Actions

Jenkins

GitLab CI

Docker

Kubernetes

Job description

Role overview

A senior Software Development Engineer in Test (SDET) role partnering with an AI engineering team at a cybersecurity software company to shape quality engineering for AI-powered products. The work blends traditional software testing with non-deterministic AI/ML validation, building automated frameworks and shifting quality left of release cycles. The position is fully remote, with a hybrid schedule available for candidates based in the El Dorado Hills, CA or Chicago, IL metro areas.

Responsibilities

• Collaborate with AI engineers, software engineers, and product managers during design to define test strategy, acceptance criteria, and automation requirements
• Design and maintain automated test frameworks for AI-powered applications, backend services, and APIs
• Develop evaluation strategies that account for both deterministic software behavior and variable AI/ML outputs
• Define measurable quality thresholds, tolerances, and evaluation criteria for AI and LLM-driven functionality
• Build regression, integration, API, and end-to-end coverage alongside feature development
• Integrate automated validation and quality gates into CI/CD pipelines

Requirements

• 5+ years in an SDET, test automation, software development, or other code-intensive quality engineering role
• Hands‑open experience testing AI/ML, LLM‑powered, or model-integrated software systems
• Strong programming skills in Python, Java, JavaScript/TypeScript, Rust, or another general-purpose language
• Demonstrated experience designing, building, and maintaining automated test frameworks
• Experience building automated functional, integration, API, and end-to-end coverage
• Experience integrating automated tests into CI/CD pipelines using tools such as GitHub Actions, Jenkins, or GitLab CI

Nice to have

• Experience testing LLMs, agentic systems, retrieval-augmented generation, tool calling, or prompt-driven workflows
• Familiarity with AI evaluation metrics for measuring quality across non-deterministic outputs
• Experience with Docker, Kubernetes, or cloud-based testing environments
• Experience testing cybersecurity, identity, privileged access, or other security-sensitive software
• Experience building shared automation frameworks or internal testing tools used across teams

Benefits and work setup

• 100% remote position; hybrid schedule available in El Dorado Hills, CA and Chicago, IL metro areas
• Medical, dental, and vision coverage inclusive of domestic partnerships
• Employer-paid life insurance plus supplemental coverage for employee, spouse, and children
• Voluntary short- and long-term disability insurance
• 401(k) plan with both Roth and Traditional options
• Generous paid time off that reflects commitment and seniority, including paid bereavement and jury duty leave