GenAI QA Automation Engineer | AI Testing & Reliability

Anaplan; Charlotte (NC); On-site; USD 120,000 - 160,000; Full time

Job summary

Anaplan is seeking a QA Automation Engineer focusing on GenAI/Agentic systems in Charlotte, NC. You will develop automated tests, evaluate GenAI features, and ensure reliability across LLM-powered workflows.

Bring 8+ years in QA, Python expertise, and strong knowledge of testing APIs, UI, and performance. Join a team committed to a Winning Culture and customers’ success with AI-driven platforms.

Qualifications

• Minimum 8 years in QA or test engineering with AI/ML exposure.
• Hands-on Python coding experience and test automation expertise.
• Knowledge of GenAI, LLM behavior, and AI testing patterns.

Responsibilities

• Develop end-to-end automated test coverage for evaluations and APIs.
• Design testing strategies for GenAI features and agentic workflows.
• Maintain automated test suites for prompt testing and regression checks.
• Support evaluation frameworks for GenAI performance (accuracy, safety, latency).
• Curate test datasets and golden examples for diverse scenarios.
• Assist with adversarial testing and production alerting integration.
• Document procedures and known issues for reproducibility.

Skills

QA Experience

Python

GenAI basics

Test Automation

CI/CD

API Testing

LLMs

Tools

Postman

REST Assured

Pytest

Job description

Anaplan is seeking a QA Automation Engineer focusing on GenAI/Agentic systems in Charlotte, NC. You will develop automated tests, evaluate GenAI features, and ensure reliability across LLM-powered workflows.

Bring 8+ years in QA, Python expertise, and strong knowledge of testing APIs, UI, and performance. Join a team committed to a Winning Culture and customers’ success with AI-driven platforms.