Senior Remote AI Quality & Test Engineer

Simple Technology Solutions, LLC; Oregon; Hybrid; USD 120,000 - 160,000; Full time

Benefits offered by this job

Competitive compensation

Health, dental, and vision coverage

Professional development and training

Performance-based incentives

401(k) retirement plan

Work from home stipend

Job summary

Simple Technology Solutions is seeking a senior QA/Test Automation Engineer for AI projects. The role emphasizes building automated CI/CD testing, UAT leadership, and evaluating AI features on AWS Bedrock. Candidates should have 8+ years in software quality and strong Python/TypeScript skills.

Remote position with occasional travel; HUBZone background preferred. You will mentor teammates and shape testing standards while partnering with federal clients.

Qualifications

• Bachelor's degree in CS or related field required.
• Minimum 8 years of software quality engineering or test automation experience.
• 2+ years testing applications built on AI/ML components.
• Experience building automated CI/CD test suites with quality gates.
• Experience leading user-acceptance testing in federal or regulated environments.
• Strong Python or TypeScript test automation using Playwright, pytest or Jest and API testing.
• Familiarity with security testing basics and federal compliance expectations.

Responsibilities

• Define the test strategy for AI projects, including unit, integration, end-to-end, UAT, security, and AI-behavior testing.
• Design, build, and execute end-to-end tests across DEV, TST, STG, and PRD.
• Build evaluation harnesses for AI behavior with datasets and rubric-based scoring.
• Operate as tester in AI development workflow; improve shared testing agents.
• Automate tests in the CI/CD pipeline and own quality gates between environments.
• Validate acceptance criteria with business analysts and real users; capture evidence.
• Triage, reproduce, and track defects in Jira; drive resolution with engineers.
• Test on AWS-native AI stacks (Bedrock) when required; assess performance and cost.
• Report findings and recommendations; contribute standards and tooling to SDLC.
• Coach the team on test-driven development and AI evaluation practices.

Skills

Python

TypeScript

Playwright

pytest

Jest

CI/CD

Test automation

LLM evaluation

Education

Bachelor's degree in Computer Science or related field

Tools

GitHub

Jira

CI/CD pipelines

Amazon Bedrock

AWS services

Job description

Simple Technology Solutions is seeking a senior QA/Test Automation Engineer for AI projects. The role emphasizes building automated CI/CD testing, UAT leadership, and evaluating AI features on AWS Bedrock. Candidates should have 8+ years in software quality and strong Python/TypeScript skills.

Remote position with occasional travel; HUBZone background preferred. You will mentor teammates and shape testing standards while partnering with federal clients.