Lead AI Engineer – Agentic Test Automation

Randstad Digital; United States; On-site; USD 90,000 - 124,000; Full time

Job summary

Randstad Digital in McLean, VA is seeking a contract-based test automation professional to lead agentic GenAI testing initiatives. You will craft reusable patterns, reference implementations, and governance artifacts across UI, API, and service layers, enabling scalable automation across teams.

The role emphasizes designing standard architectures, coverage templates, and GenAI-driven summaries, with CI/CD integration and governance to ensure quality and speed in delivery.

Qualifications

• GenAI / LLM agentic development
• Hands-on experience building LLM-powered agents (tool-using, multi-step reasoning, guardrails).
• Experience with prompting patterns, structured outputs (JSON schemas), evaluation, and reducing hallucinations.
• Ability to design agent workflows for: Test generation/augmentation, Requirements review and completeness validation, Report generation and summarization.
• Strong proficiency with GitHub Copilot in day-to-day development.
• Deep experience with GitHub platform capabilities: GitHub Actions, PR checks, branch protections, CODEOWNERS, templates, and APIs/webhooks.
• Test automation engineering: framework expertise; automation with Karate and Playwright.
• Cross-service reporting and data aggregation; automated reports and trend analysis.
• Proven ability to aggregate and normalize results from multiple microservices and pipelines.

Responsibilities

• Agentic test automation foundation with reusable patterns reference implementations.
• Design and implement agentic testing patterns adopted by underwriting teams and later domains.
• Create reference implementations demonstrating test generation, maintenance, and failure analysis.
• Establish standard architecture for test code organization, tagging, data management, and execution across UI/API/service layers.
• Define and publish coverage standards, templates, and governance for testing.
• Provide templates for test plans, test case/specs, and Definition of Ready/Done checklists.
• Create scalable tagging/metadata strategy to support reporting and quality gates.
• GenAI-assisted reporting across microservices; dashboards and markdown summaries in PRs.
• Develop automated gates to evaluate user stories/requirements for data clarity before development/testing.

Skills

GenAI / LLM agentic development

Hands-on LLM-powered agents

Prompting patterns

GitHub Copilot

GitHub Actions

Karate

Playwright

Test automation engineering

Test design & coverage

Cross-service data aggregation

Education

Bachelor's degree

Tools

Karate

Playwright

GitHub Copilot

GitHub Actions

GitHub APIs/webhooks

Job description

Job Summary

Fully onsite McLean, VA

location: Tysons, Virginia

job type: Contract

salary: $65 – 90 per hour

work hours: 9am to 5pm

education: Bachelors

Responsibilities

1) Agentic test automation foundation (reusable patterns reference implementations)

Design and implement agentic testing patterns that can be adopted by multiple Underwriting teams (and later other domains).

Create reference implementations (sample repos / templates) demonstrating:

• Test generation assistance (from requirements, APIs, contracts, schemas)
• Test maintenance assistance (auto-updating selectors/contracts, flaky test triage)
• Failure analysis assistance (root cause suggestions, log correlation, defect drafting)

Establish a standard architecture for test code organization, tagging, data management, and execution across UI API service layers.

2) Coverage standards, templates, and governance

• Define and publish coverage standards (what “good” looks like) including:
  • Minimum coverage expectations by service/component
  • Test type mix (unit vs API vs UI vs contract vs integration)
  • Risk-based prioritization and traceability to requirements
• Provide templates usable across teams:
  • Test plan templates
  • Test case/spec templates (Gherkin-style or equivalent)
  • Definition of Ready / Definition of Done quality checklists
• Create a scalable tagging/metadata strategy (e.g., feature, service, risk, priority, data sensitivity) to support reporting and quality gates.

3) GenAI-assisted reporting and quality insights across microservices

• Build automated reporting that aggregates test service data across multiple microservices, such as:
  • Test execution results (Karate/Playwright CI runs)
  • Service health signals (logs/metrics/traces if available)
  • Defect signals (issue tracker metadata if available)
• Generate GenAI-driven summaries:
• Release readiness narratives
• Failure clustering and trend analysis
• “What changed?” insights (commit/PR correlation)
• Produce outputs consumable by engineering leadership and teams (dashboards, markdown summaries in PRs, artifacts in CI).

4) “Quality gates” via agents

• Build automated review agents that evaluate user stories/requirements for minimum required clarity and data before development/testing starts:
  • Required fields present (acceptance criteria, testable outcomes, data needs, dependencies)
  • Ambiguity detection and missing edge cases
  • Data/privacy considerations and environment needs
• Integrate gates into workflow (PR checks, issue templates, GitHub Actions) to reduce churn and rework.

Qualifications

Required Technical Skills (must-have)

GenAI / LLM agentic development

Hands-on experience building LLM-powered agents (tool-using, multi-step reasoning, guardrails).

Experience with prompting patterns, structured outputs (JSON schemas), evaluation, and reducing hallucinations.

Ability to design agent workflows for:

• Test generation/augmentation
• Requirements review and completeness validation
• Report generation and summarization

GitHub platform GHCP (Copilot) for engineering workflows

Strong proficiency with GitHub Copilot in day-to-day development.

Deep experience with GitHub platform capabilities:

• GitHub Actions (CI/CD pipelines, reusable workflows, composite actions)
• PR checks, branch protections, CODEOWNERS, templates
• Automation via GitHub APIs/webhooks (as needed)

Test automation engineering (framework expertise)

Advanced experience designing and implementing automation with:

• Karate (API testing, contract-like checks, data-driven testing, mocks)
• Playwright (UI automation, selectors strategy, parallelization, trace/video artifacts)

Strong understanding of test design and coverage:

• Happy path scenarios
• Negative/validation scenarios
• Edge/boundary scenarios
• Data setup/teardown strategies and test isolation

Cross-service reporting and data aggregation

Proven ability to aggregate and normalize results from multiple microservices and multiple pipelines.

Experience producing actionable automated reports (trend analysis, failure clustering, service correlation).

Automated requirements review agents

Experience implementing automated checks that validate:

• Acceptance criteria completeness
• Required test data and environment dependencies
• Non-functional requirements (performance, security, observability) when applicable

Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.

At Randstad Digital, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact

Pay offered to a successful candidate will be based on several factors including the candidate’s education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad Digital offers a comprehensive benefits package, including: medical, prescription, dental, vision, AD&D, and life insurance offerings, short-term disability, and a 401K plan (all benefits are based on eligibility).

This posting is open for thirty (30) days.