Quality Assurance Engineer (Remote)

Thompson Street Capital Partners; Oklahoma City (OK); Remote; USD 90,000 - 120,000; Full time

Job summary

RMS in Oklahoma City is seeking a Quality Assurance (QA) Engineer to design, build, and maintain automated test frameworks for web apps and AI-driven features. You will perform UI/API testing and validate non-deterministic outputs from ML models to ensure reliability and performance.

You will containerize test environments with Docker, integrate tests into CI/CD pipelines (GitHub Actions, Azure DevOps, Jenkins), and collaborate with Product Owners, Software Engineers, and Data/ML Engineers to

Qualifications

• Hands-on experience with modern web automation and end-to-end testing tools (Playwright/Cypress/Selenium/WebdriverIO).
• Docker-based test environments and CI/CD integration experience.
• Proficiency in API testing using Postman or REST Assured.
• Solid knowledge of HTML5/CSS3/JavaScript/TypeScript and modern web architectures.
• Strong coding skills in at least one modern language (TypeScript/JavaScript, Python, C#, or Java).
• Proficiency with SQL and querying relational databases for test setup and verification.

Responsibilities

• Design, develop, and maintain scalable automation frameworks for modern SPAs and microservices.
• Integrate automated tests into CI/CD pipelines (GitHub Actions, Azure DevOps, Jenkins).
• Create and manage containerized test environments with Docker.
• Test AI/ML features, validate non-deterministic outputs and model safety/bias guardrails.
• Collaborate with Product, Software, and Data/ML Engineers during Agile/Scrum sprints.
• Conduct exploratory, regression, performance, and cross-browser testing.

Skills

Automation frameworks

CI/CD

AI/ML testing

Debugging

Cross-functional collaboration

Education

Bachelor's degree preferred

Tools

Playwright

Cypress

Selenium

WebdriverIO

Docker

Postman

Job description

Job Description

The Quality Assurance (QA) Engineer plays a pivotal role in ensuring the quality, reliability, and performance of our web applications and AI-driven solutions. This role is responsible for designing, building, and maintaining automated test frameworks, conducting hands-on exploratory and UI/API testing, and validating non-deterministic outputs from machine learning models.

In addition to traditional QA practices, you will leverage modern containerization tools (such as Docker) to manage isolated test environments and utilize AI-assisted tools (such as Copilot, synthetic test data generators, and intelligent test execution platforms) to optimize testing workflows, improve test coverage, and accelerate release cycles.

Duties and Responsibilities

• Modern Web & API Automation:

• Design, develop, and maintain robust, scalable automation frameworks for modern single-page web applications (React, Angular, Vue) and microservices/REST APIs.


• Integrate automated test suites into CI/CD deployment pipelines (e.g., GitHub Actions, Azure DevOps, Jenkins) for continuous validation.

• Containerized Test Environments & Infrastructure:

• Spin up, manage, and tear down local and cloud-based test environments using Docker containers and container orchestration tools.


• Execute automated test suites inside containerized environments to ensure consistency across local, staging, and production-like setups.

• AI & Model Output Testing:

• Develop test strategies to validate AI/ML features, large language model (LLM) outputs, and non-deterministic application behavior for accuracy, relevancy, latency, and edge cases.


• Monitor and measure AI output consistency, safety/bias guardrails, and system performance under varying prompt payloads.

• AI-Driven QA Optimization:

• Utilize AI-assisted development tools and intelligent QA platforms to generate test cases, synthesize realistic test data sets, and accelerate root-cause failure analysis.

• Cross-Functional Collaboration:

• Collaborate closely with Product Owners, Software Engineers, and Data/ML Engineers during Agile/Scrum sprints to refine requirements, acceptance criteria, and edge-case scenarios.


• Participate in code reviews for test automation scripts to ensure high code quality and maintainability.

• Quality Process & Defect Management:

• Identify, document, prioritize, and track defects using modern issue-tracking tools (e.g., Jira).


• Perform exploratory, regression, performance, and cross-browser/device testing across staging and production environments.


• Assist in setting QA standards, test metrics, and best practices across the engineering organization.

Qualifications

Core Technical Requirements

• Automation Frameworks: Hands-on experience with modern web automation and end-to-end testing tools (e.g., Playwright, Cypress, Selenium, or WebdriverIO).


• Containerization & DevOps: Solid practical knowledge of Docker (writing Dockerfiles, using Docker Compose) for spinning up containerized applications, databases, and automated testing nodes.


• API Testing: Proficiency in API testing and automation using tools like Postman, REST Assured, or HTTP clients.


• Modern Web Technologies: Solid understanding of modern web architectures (HTML5, CSS3, JavaScript/TypeScript, DOM manipulation, asynchronous state management).


• Programming & Scripting: Strong coding skills in at least one modern language (e.g., TypeScript/JavaScript, Python, C#, or Java).


• Database Querying: Proficiency with SQL (PostgreSQL, SQL Server, MySQL) and experience querying relational or document databases for test setup and verification.
  

AI & Emerging Tech Capabilities

• AI Feature Testing: Familiarity with concepts around evaluating AI/ML features, RAG architectures, prompt engineering testing, or heuristic evaluation metrics.


• AI Productivity Tools: Familiarity or experience using AI tools (e.g., GitHub Copilot, ChatGPT, Claude) to write test scripts, create mock data, or generate test scenarios
  

Professional & Interpersonal Skills

• Agile Methodology: Proven experience working in an Agile/Scrum environment.


• Problem-Solving: Exceptional analytical, critical thinking, and debugging skills with high attention to detail.


• Communication: Excellent written and verbal communication skills to articulate technical defects, risk assessments, and test coverage clearly to technical and non-technical stakeholders.

Education/Experience

A bachelor's degree is not required, but preferred. More than anything, RMS is looking for people who are passionate about technology who want to develop the skills to solve hard and important problems.

Environmental Conditions

Indoor climate-controlled environment. Moderate to quiet noise level

Physical requirements

While performing the duties of this Job, the employee is regularly required to communicate verbally and in the written form. The employee is physically required to utilize a laptop and other electronic devices effectively. The employee must lift and/or move up to 20 pounds (laptop computer, bag, and accessories). Specific vision abilities required by this job include close vision and distance vision.

All applicants are subject to drug screens and background checks per company policies.