Senior SDET - GenAI & AI Quality Assurance

Anaplan; Charlotte (NC); On-site; USD 150,000 - 210,000; Full time

Job summary

Anaplan is hiring a Senior SDET for AI to own testing for GenAI-powered features. You will design test plans, build end-to-end automated suites in Python, and develop prompt testing systems to guard model behavior.

You will work across evaluations, API checks, UI tests, and performance tests with a focus on accuracy, safety, and latency. Join Anaplan’s Winning Culture and help ensure our AI features are reliable, accurate, and trustworthy across production environments.

Qualifications

• Design test plans and evaluation frameworks for LLM-powered applications.
• Develop end-to-end automated test suites in Python across evaluations, API checks, UI checks, and performance tests.
• Create automated prompt testing systems and regression suites to detect model behavior changes.
• Establish multi-dimensional evaluation criteria for GenAI quality: accuracy, relevance, safety, consistency, latency.
• Perform adversarial testing to identify and mitigate hallucinations, biases, or security issues.
• Develop internal tools and testing frameworks to help SE teams validate GenAI implementations.
• Deploy real-time monitoring and alerting to detect quality degradation in production.
• Own the overall quality and testing strategy for your workstream, maintaining goldens and QA best practices.

Responsibilities

• Design and architect test strategies for GenAI-powered workflows.
• Build automated test suites across evaluations, API, UI, and performance testing.
• Create prompt testing systems and regression suites to track model behavior.
• Define evaluation criteria for GenAI quality—accuracy, safety, latency, relevance.
• Conduct adversarial testing to uncover vulnerabilities in models.
• Develop internal QA tools to support other teams’ GenAI deployments.
• Set up real-time monitoring and alerting for production quality.

Skills

GenAI testing

Python

Selenium

Pytest

Test automation

API testing

CI/CD

Non-deterministic testing

Problem solving

Tools

Postman

REST Assured

Job description

Anaplan is hiring a Senior SDET for AI to own testing for GenAI-powered features. You will design test plans, build end-to-end automated suites in Python, and develop prompt testing systems to guard model behavior.

You will work across evaluations, API checks, UI tests, and performance tests with a focus on accuracy, safety, and latency. Join Anaplan’s Winning Culture and help ensure our AI features are reliable, accurate, and trustworthy across production environments.