Senior QA Engineer - AI & Agentic Systems (Remote)

GCS Recruitment; United States; Remote; USD 120,000 - 160,000; Full time

Job summary

GCS Recruitment is seeking a Senior IC QA Automation Engineer to lead testing of agentic AI systems. The role emphasizes non-deterministic evaluation, tool-use testing, and ensuring outputs stay within defined guardrails across integration, API, and performance testing.

You will design and run frameworks with independence and own high-quality verification across live production and test environments. The candidate should have extensive Python experience, be comfortable with performance testing

Qualifications

• 5+ years in QA automation or quality engineering, with ML/AI testing focus.
• Experience testing ML models, LLM applications, or AI agents.
• Hands-on with evaluation frameworks and automated test harnesses.

Responsibilities

• Tool-Use & Trajectory Evaluation: verify tool choices and multi-step reasoning.
• Grounding & Citation Verification: ensure evidence supports claims.
• Honest-Failure & Refusal Calibration: ensure proper handling of uncertain info.
• Guardrail & Adversarial Testing: probe prompts and hidden instructions.
• Multi-Turn & State: validate session continuity and references across turns.
• Backend, UI & API Testing: build robust integration tests across services.

Skills

Python

Selenium/Playwright

Pytest

Locust

JMeter

k6

Git/GitLab

OpenTelemetry

LangSmith/Langfuse

Tools

Requests

Job description

GCS Recruitment is seeking a Senior IC QA Automation Engineer to lead testing of agentic AI systems. The role emphasizes non-deterministic evaluation, tool-use testing, and ensuring outputs stay within defined guardrails across integration, API, and performance testing.

You will design and run frameworks with independence and own high-quality verification across live production and test environments. The candidate should have extensive Python experience, be comfortable with performance testing