QA Engineer

Dataservtech; Andover (MA); On-site; USD 85,000 - 110,000; Full time

Benefits offered by this job

Paid holidays

PTO

Discretionary bonuses

401K with match

Medical

Dental

Vision

Life

STD

LTD

Job summary

DataServ/xFact is seeking a QA Engineer to act as the QA SME for testing automation across web, mobile, and backend products, including AI/LLM features. You will help shape testing workflows for AI-driven capabilities and ensure quality and reliability across our enterprise solutions.

Join a collaborative, mission-driven team with a strong focus on meaningful work, flexible benefits, and opportunities for internal growth as we expand our AI testing and evaluation capabilities.

Qualifications

• Bachelor’s degree in computer science or related QA field.
• 2+ years in Software Quality Assurance.
• Experience designing and executing manual and automated tests.
• Proficiency with at least one scripting language (Python, JavaScript/TypeScript, or Java).

Responsibilities

• Test planning, design, and execution for web, mobile, and backend features.
• Develop and maintain automated tests (Selenium, Playwright, Cypress) using Python/JS.
• Run regression and release testing before deployments.
• Identify, document, and track defects with Jira; work with developers to resolve.
• Integrate tests into CI/CD pipelines (GitLab CI) and improve QA processes.

Skills

Team player

Excellent communicator

Tech savvy

Strategic thinker

Service oriented

Motivated

Self-disciplined

Professional

Fast-paced

Independent worker

Detail oriented

Deadline oriented

Organized

Dependable

Feedback oriented

Mentor mindset

Education

Bachelor’s degree in computer science

Tools

Selenium

Playwright

Cypress

Python

JavaScript

TypeScript

Java

TestRail

Jira

Git

Job description

Description

DataServ/xFact is seeking a QA Engineer to serve as the subject matter expert for Quality Assurance. You will test automation for our enterprise web, mobile, and backend products, and help evaluate and improve AI/LLM-powered features, including how they’re tested, measured, and released. You’ll ensure the quality and reliability of our products while helping define and execute the crucial testing and evaluation workflows for our AI-driven features.

About xFact/DataServ:

xFact and DataServ bring together nearly 40 years of experience delivering innovative technology solutions to public and commercial sectors. We combine a strong foundation in Technology with a fast growing, mission-driven team focused on building impactful solutions for our existing and growing customer base that spans the country. Our culture is collaborative, driven, and focused on doing meaningful work that creates real value.

Qualities we want in you:

• team player
• excellent written and verbal communicator
• tech savvy
• strategic thinker
• service oriented
• motivated
• self-disciplined
• professional
• flexible in a fast-paced environment
• works independently with minimal supervision
• exhibits strong work ethics
• detail oriented
• handles demanding situations/timelines
• organized
• responsible/dependable
• willing to receive/provide feedback to improve performance
• supportive
• strong desire to support and mentor

What you’ll do:

Quality Assurance & Test Automation

• Test Planning & Execution - Design, develop, and execute manual and automated test plans and test cases for enterprise features across web, mobile, and backend systems
• Automation Development - Develop and maintain automated test scripts (e.g., Selenium, Playwright, Cypress) using a scripting/programming language such as Python or JavaScript to increase efficiency and coverage
• Regression & Release Testing - Run regression suites before releases to ensure existing functionality remains stable as new features and models are deployed
• Defect Management - Identify, document, prioritize, and track defects using tools like Jira, and collaborate with developers to drive timely resolution
• CI/CD & Process Improvement - Integrate tests into CI/CD pipelines (e.g. GitLab CI) and participate in continuous improvement of QA processes, tools, and methodologies

AI & LLM Evaluation

• Evaluation Planning - Partner with Product Managers and Data Scientists to define scope, metrics, and goals for evaluating AI and LLM-powered features (e.g., accuracy, robustness, fairness, hallucination rate, safety)
• Data Curation & Preparation - Coordinate collection, labeling, and preparation of datasets used for AI model testing and human-in-the-loop evaluations
• Evaluation Execution & Coordination - Help organize evaluation sprints, red-teaming sessions, and feedback loops, acting as a point of contact between AI/ML, Product, QA, and external evaluators (if applicable)
• Results Analysis & Reporting - Compile and summarize evaluation results, providing clear, actionable feedback on model performance and readiness for production deployment
• Documentation & Traceability - Maintain detailed documentation of evaluation criteria, test data, prompts, and results, ensuring traceability and auditability for AI/LLM releases

AI for QA (Tooling & Enablement)

• Use LLMs and AI tools (e.g., ChatGPT, Copilot, AI testing platforms) to generate test ideas, draft test cases, and assist with log/defect analysis, then validate and refine these outputs
• Help evaluate and adopt AI-powered QA tools (e.g., self-healing locators, intelligent test selection, test impact analysis)
• Contribute to best practices and guardrails for using AI safely and effectively in QA and evaluation workflows

What we require:

Education

• Bachelor’s degree in computer science, or related professional experience in Quality Assurance

Experience

• 2+ years in Software Quality Assurance or a similar role.

Skills

• Experience designing and executing manual and automated tests
• Proficiency with at least one scripting/programming language (e.g., Python, JavaScript/TypeScript, or Java)

Tools

• Familiarity with test management tools (e.g., TestRail or similar)
• Defect tracking systems (e.g., Jira)
• Version control (e.g., Git)

Additional proficiencies we seek:

• Demonstrates excellent written and verbal communication skills, in both collaborative team and customer-facing environments
• Candidates are required to complete a drug test and background check

Why you’ll love us:

We care. We coach. We lead. We continue to use this approach in all we do for our culture. It’s truly the people who make working here great. We are a collaborative environment and supportive of one another, providing flexibility when needed. As lifelong learners, promoting from within is a priority to provide our team members opportunities for internal mobility and growth. DataServ/xFact offer a competitive compensation plan including paid holidays, PTO, discretionary bonuses, 401K (with match), medical, dental, vision, life, STD, and LTD benefits along with many other inclusive benefits and perks!

DataServ/xFact is an equal opportunity employer and does not discriminate in hiring or employment, in accordance with the requirements of all applicable laws, including race, color, religion, national origin, ancestry, age, gender, marital status, military status, sexual orientation, disability, or medical condition.