AI/ML Testing Lead — QA for Generative Models

TekWissen LLC; Bellevue (WA); Hybrid; USD 120,000 - 160,000; Full time

Job summary

TekWissen Group is seeking a Lead II - Software Testing in Bellevue, WA for a 6-month hybrid assignment. You will design and execute test plans for AI/ML features, build automated test suites, and evaluate model outputs for accuracy, bias, and robustness.

Collaboration with data scientists and ML engineers is essential to define acceptance criteria and quality metrics. The role requires strong Python, SQL, and experience with eval frameworks.

Qualifications

• Experience designing and executing test plans for AI/ML features.
• Strong Python scripting and data validation skills.
• Familiarity with eval metrics for generative AI.

Responsibilities

• Design and execute test plans for AI/ML-driven features, including model outputs, prompts, and integrated application behavior.
• Build and maintain automated test suites covering functional, regression, integration, and API testing.
• Evaluate model outputs for accuracy, consistency, bias, hallucination, and edge-case failures.
• Develop evaluation frameworks and golden datasets/test cases to benchmark model performance over time.
• Test prompt engineering changes, model version upgrades, and fine-tuning outputs for regressions.
• Perform adversarial and red-team style testing to surface safety, security, and robustness issues.

Skills

AI/ML testing

Python

Test automation

SQL

CI/CD

Adversarial testing

Tools

Ragas

DeepEval

LangSmith

Promptfoo

OpenAI Evals

TruLens

Job description

TekWissen Group is seeking a Lead II - Software Testing in Bellevue, WA for a 6-month hybrid assignment. You will design and execute test plans for AI/ML features, build automated test suites, and evaluate model outputs for accuracy, bias, and robustness.

Collaboration with data scientists and ML engineers is essential to define acceptance criteria and quality metrics. The role requires strong Python, SQL, and experience with eval frameworks.