Test Automation Lead / Architect

Randstad Digital Americas; Houston (TX); On-site; USD 110,000 - 120,000; Full time

Job summary

Randstad Digital Americas in Houston seeks a Test Automation Lead/Architect to define and lead an enterprise-grade automation strategy across UI, API, data, and platform layers, aligning with product risk and release cadence.

You will architect scalable frameworks, drive engineering excellence, and accelerate delivery through high-productivity automation programs while guiding QE teams in AI/LLM quality assurance and governance.

Qualifications

• 8-12+ years in Quality Engineering / Test Automation with architect-level ownership.
• Experience designing scalable automation across API/UI/contract/integration/e2e.
• Strong programming skills in Java, Python, TypeScript/JavaScript, or C#.

Responsibilities

• Define and own the test automation architecture and multi-layer strategy (UI/API/contract/integration/e2e).
• Establish automation standards, patterns, and reference implementations.
• Drive shift-left testing and reduce reliance on fragile end-to-end tests.
• Design testing approaches for AI outputs, including confidence thresholds and evaluation harnesses.
• Lead QE technical authority: coach engineers on automation design and code quality.
• Improve CI/CD pipelines and quality gates with automated dashboards.

Skills

Agentic AI

API testing

UI automation

Test automation

Programming skills

Leadership

Mentorship

Quality engineering

Risk-based testing

Education

Bachelor's degree

Tools

Playwright

Cypress

Selenium

REST-assured

Postman/Newman

pytest

GitHub Actions

Jenkins

Azure DevOps

Job description

Job Summary

We are seeking a Test Automation Lead / Architect to define and lead an enterprise-grade test automation strategy across UI, API, data, and platform layers-while evolving our quality engineering capabilities to support Agentic AI / LLM-based systems (probabilistic outputs, confidence-based validation, hallucination detection, and model evaluation). This role will architect scalable automation frameworks, drive engineering excellence, and accelerate delivery through highly productive, maintainable automation programs.

location: Houston, Texas

job type: Permanent

salary: $110,000 - 120,000 per year

work hours: 8am to 4pm

education: Bachelors

Responsibilities

Automation Strategy & Architecture

• Define and own the test automation architecture and multi-layer strategy (UI/API/contract/integration/e2e), aligned to product risk and release cadence.
• Establish automation standards, patterns, and reference implementations (framework structure, page-object/screenplay usage, test data patterns, reporting).
• Drive shift-left testing practices (contract testing, component testing, pre-merge validation) and reduce reliance on fragile end-to-end tests.

Agentic AI / LLM Test Model Alignment

• Design and implement testing approaches for non-deterministic AI outputs, including:
  • confidence/threshold-based assertions
  • semantic similarity checks and rubric-based validation
  • goldens/benchmark datasets and evaluation harnesses
  • hallucination and groundedness testing strategies
  • regression detection across model/prompt/version changes
• Define AI test coverage across accuracy, robustness, bias/safety (as applicable), performance, and reliability.
• Partner with ML/AI, Product, and Engineering to establish acceptance criteria for AI capabilities and measurable quality gates.

High-Productivity Automation Programs

• Build and scale high-throughput automation projects that are reliable, fast, and easy to extend.
• Improve test execution time, reduce flaky tests, and optimize CI/CD pipelines (parallelization, test selection, retries with guardrails).
• Create actionable, engineering-friendly quality dashboards (pass/fail trends, flake rate, coverage, defect escape metrics).

Tooling, CI/CD, and Quality Gates

• Integrate automation into CI/CD with quality gates (PR checks, nightly suites, release readiness criteria).
• Standardize observability for testing: logs, tracing hooks, screenshots/video, artifacts, and test run diagnostics.
• Define practices for test environments, data management, service virtualization, and dependency control.

Technical Leadership & Mentorship

• Serve as the QE technical authority: coach engineers on automation design, code quality, and best practices.
• Lead design reviews for test architecture and automation PRs; enforce coding standards and maintainability.
• Influence roadmap and delivery by translating risk into clear testing priorities.

Qualifications

Required Qualifications

8-12+ years in Quality Engineering / Test Automation with architect-level ownership of frameworks and strategy.

Proven Experience Designing Scalable Automation Across

• API testing (contract + integration)
• UI automation with stable locator strategies (test IDs, accessibility roles)
• Test data strategy and environment reliability
• Strong programming skills in at least one modern language (e.g., Java, Python, TypeScript/JavaScript, C#).

Expertise With Automation Tools/frameworks (examples)

• UI: Playwright / Cypress / Selenium
• API: REST-assured / SuperTest / Postman/Newman / pytest
• CI: GitHub Actions / Jenkins / Azure DevOps
• Demonstrated ability to reduce flakiness and improve execution speed through engineering solutions.

Preferred Qualifications (Strongly Desired)

• Experience testing LLM/Agentic AI features or other probabilistic systems, including:
  • evaluation harnesses, benchmark datasets, semantic scoring, and drift detection
  • prompt/version regression strategies and structured evaluation reports
• Familiarity with modern quality approaches: test pyramid, contract testing, consumer-driven contracts, risk-based testing, shift-left.
• Knowledge of security/performance testing integration (baseline understanding; hands-on is a plus).
• Experience building reusable automation libraries for multiple teams/products.

Skills

skills: Agentic AI,API testing,API,AI,AI capabilities,Test Automation,C#,test coverage,Strong programming skills,CI/CD,Cypress,data management,LLM-based systems,GitHub Actions,hallucination detection,IDs,Java,JavaScript,Jenkins,LLM,Azure DevOps,UI automation,model evaluation,performance testing,Playwright,Postman,pytest,Python,REST-assured,Selenium,service virtualization,code quality,testing strategies,test data,test execution,TypeScript,component testing,friendly,productive,reliability,acceptance criteria,Architecture,scalable automation,automation,automation tools,dashboards,shift-left testing,diagnostics,innovative,probabilistic,Mentorship,metrics,automation projects,quality engineering,risk,risk-based testing,rubric,safety,security,semantic,Standardize,Technical Leadership,Automation Strategy,testing,Tooling

Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.

At Randstad Digital, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact [e-mail].

Pay offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad Digital offers a comprehensive benefits package, including: medical, prescription, dental, vision, AD&D, and life insurance offerings, short-term disability, and a 401K plan (all benefits are based on eligibility).

This posting is open for thirty (30) days.