GenAI Developer

Technology Ventures; McLean (VA); On-site; USD 120,000 - 190,000; Full time

Job summary

Technology Ventures seeks a Senior AI Testing Architect to lead agentic testing initiatives across UI, API, and service layers. You will design patterns, implement reference templates, and enable teams with automated testing powered by GenAI, Copilot, and Playwright.

Responsibilities include creating standards for coverage, templates for test cases, and automated quality gates integrated into GitHub workflows to reduce churn and accelerate delivery.

Qualifications

• 5+ years of experience and strong AI development background.
• Hands-on experience building agentic workflows and automating testing with AI.
• Experience with tools like GitHub, Copilot, and Claude.

Responsibilities

• Design and implement agentic testing patterns adopted across teams.
• Create reference implementations and templates for test generation, maintenance, and failure analysis.
• Build automated reporting and quality insights across microservices.

Skills

GenAI / LLM

Agentic workflows

AI-driven testing

GitHub Copilot

Playwright

UI automation

Test design

Data setup/teardown

GitHub APIs

Tools

GitHub

GitHub Actions

Copilot

Job description

Must Have Qualifications

Must have 5+ years of experience and a strong AI development background. Must have hands on experience building agentic workflows, automating testing using AI, and working with tools such as GitHub, Copilot, and Claude.

Design and implement agentic testing patterns that can be adopted by multiple Underwriting teams (and later other domains).

Create reference implementations (sample repos / templates) demonstrating:

• Test generation assistance (from requirements, APIs, contracts, schemas)
• Test maintenance assistance (auto-updating selectors/contracts, flaky test triage)
• Failure analysis assistance (root cause suggestions, log correlation, defect drafting)

Establish a standard architecture for test code organization, tagging, data management, and execution across UI + API + service layers.

2) Coverage standards, templates, and governance

• Define and publish coverage standards (what "good" looks like) including:
• Minimum coverage expectations by service/component
• Test type mix (unit vs API vs UI vs contract vs integration)
• Risk-based prioritization and traceability to requirements

• Provide templates usable across teams:
• Test case/spec templates (Gherkin-style or equivalent)
• Definition of Ready / Definition of Done quality checklists

3) GenAI-assisted reporting and quality insights across microservices

• Build automated reporting that aggregates test + service data across multiple microservices, such as:
• Service health signals (logs/metrics/traces if available)
• Defect signals (issue tracker metadata if available)
• Release readiness narratives
• Failure clustering and trend analysis
• "What changed?" insights (commit/PR correlation)

Produce outputs consumable by engineering leadership and teams (dashboards, markdown summaries in PRs, artifacts in CI).

4) "Quality gates" via agents

• Build automated review agents that evaluate user stories/requirements for minimum required clarity and data before development/testing starts:
• Required fields present (acceptance criteria, testable outcomes, data needs, dependencies)
• Ambiguity detection and missing edge cases
• Data/privacy considerations and environment needs

Integrate gates into workflow (PR checks, issue templates, GitHub Actions) to reduce churn and rework.

Required Technical Skills (must-have)

GenAI / LLM + agentic development

• Hands-on experience building LLM-powered agents (tool-using, multi-step reasoning, guardrails).
• Experience with prompting patterns, structured outputs (JSON schemas), evaluation, and reducing hallucinations.
• Ability to design agent workflows for:

• Requirements review and completeness validation
• Report generation and summarization

GitHub platform + GHCP (Copilot) for engineering workflows

• Strong proficiency with GitHub Copilot in day-to-day development.
• Deep experience with GitHub platform capabilities:

• GitHub Actions (CI/CD pipelines, reusable workflows, composite actions)
• PR checks, branch protections, CODEOWNERS, templates

• Automation via GitHub APIs/webhooks (as needed)
• Advanced experience designing and implementing automation with Playwright (UI automation, selectors strategy, parallelization, trace/video artifacts)

• Strong understanding of test design and coverage:

• Data setup/teardown strategies and test isolation

Cross-service reporting and data aggregation

• Proven ability to aggregate and normalize results from multiple microservices and multiple pipelines.
• Experience producing actionable automated reports (trend analysis, failure clustering, service correlation).

Automated requirements review agents

• Experience implementing automated checks that validate:

• Required test data and environment dependencies
• Non-functional requirements (performance, security, observability) when applicable

Deliverables / What success looks like (for the posting)

• A reusable agentic testing automation kit adopted by multiple teams.
• Published coverage standards + templates and onboarding documentation.
• A working GenAI-assisted reporting pipeline aggregating results across microservices.
• Automated quality gates integrated into GitHub workflows that measurably reduce story churn.