AI Automation QA Engineer - Hybrid

Conditions. Workplace Diversity, LLC.; Cambridge (MA); Hybrid; USD 120,000 - 160,000; Full time

Job summary

Harvard University IT is seeking an AI Automation Quality Engineer to advance the my.harvard portal with intelligent, agent-based testing. You will design AI agents, analyze requirements and code, generate reusable test assets, and continually improve testing strategies to accelerate delivery while improving software quality.

Collaborate with business analysts, developers, and product owners to validate tests, integrate Playwright, Selenium, and pytest into CI/CD, and maintain regression suites

Qualifications

• Minimum of two years’ post-secondary education or relevant work experience.
• 3+ years building automated test frameworks for enterprise web applications.
• Strong Python development experience.
• Experience testing Django and/or FastAPI applications.
• Experience with Playwright, Selenium, Cypress, pytest, or similar frameworks.
• Experience using Jira and Agile development processes.
• Understanding of Git workflows and CI/CD pipelines.
• Experience working with LLMs, prompt engineering, and AI-assisted software development.
• Understanding of Retrieval-Augmented Generation (RAG), embeddings, vector databases, MCP, or agent frameworks such as LangGraph, CrewAI, AutoGen, Semantic Kernel, or similar.
• Strong analytical thinking and ability to translate business requirements into automated quality strategies.
• Knowledge of advanced IT project management principles (e.g. Agile) and software
• Knowledge of SDLC and STLC.

Responsibilities

• This role leads AI automation quality engineering for the my.harvard portal, moving beyond traditional QA to intelligent testing capabilities.
• AI-Powered Test Generation: design AI agents to read Jira stories and criteria to generate test scenarios.
• Code-Aware Testing: build agents to analyze Django, FastAPI, and frontend codebases and generate reusable tests.
• RAG & Knowledge Engineering: develop Retrieval-Augmented Generation pipelines leveraging Jira, Confluence, Git, and docs.
• Human-in-the-Loop Validation: implement workflows where generated tests are reviewed by BAs/PMs before automation code is produced.
• Autonomous Test Automation: generate maintainable automated tests using Playwright, Selenium, pytest, etc.
• Continuous Learning: capture feedback and improve prompts and testing patterns over time.
• Quality Engineering: maintain regression suites, API tests, UI automation, performance validation, and CI/CD integration.
• Collaboration: partner with developers, architects, BAs, and DevOps to define AI-driven quality standards.

Skills

Python development

Test automation

AI / LLM testing

SDLC understanding

Problem solving

Education

Post-secondary education or equivalent

Tools

Django

FastAPI

Playwright

Selenium

Cypress

pytest

Jira

Confluence

Git

CI/CD

LangGraph

CrewAI

Job description

Harvard University IT is seeking an AI Automation Quality Engineer to advance the my.harvard portal with intelligent, agent-based testing. You will design AI agents, analyze requirements and code, generate reusable test assets, and continually improve testing strategies to accelerate delivery while improving software quality.

Collaborate with business analysts, developers, and product owners to validate tests, integrate Playwright, Selenium, and pytest into CI/CD, and maintain regression suites