QA and Test Automation Expert

Vraify; Deutschland; Remote; EUR 31,000 - 44,000; Full time

Job summary

Vraify is seeking a practical, hands-on tester to stress-test AI-generated code and adversarial task scenarios, review realism, and verify outcomes with reproducible evidence. This is a remote, project-based role open to international applicants and paid up to $30/hour depending on project scope and demonstrated expertise.

You will create, solve, or review tasks around test strategy, automation frameworks, and defect analysis; write clear test references and explanations; and review peers’ work

Qualifications

• Experience designing robust test strategies for AI/ML code.
• Proven ability to work with or review automation frameworks.
• Strong focus on reproducible evidence and reliable defect analysis.

Responsibilities

• Create, solve, or review realistic tasks involving test strategy and automation.
• Test AI-generated code and agent traces for correctness and edge cases.
• Write strong tests, expected outcomes, and concise failure explanations.
• Review peer-authored tasks for realism and clear evaluation criteria.

Skills

Test strategy

Automation frameworks

Adversarial testing

Defect analysis

Job description

Help us find where advanced coding agents break in qa and test automation. Build or solve realistic tasks, stress-test generated code, and explain failures with reproducible evidence. Remote project-based work can pay up to $30/hour, depending on the project and demonstrated expertise.

What you’ll do:

• Create, solve, or review realistic tasks involving test strategy, automation frameworks, hidden and adversarial tests, reliability, and reproducible defect analysis.
• Test AI-generated code and agent traces for correctness, edge cases, regressions, unsafe assumptions, and incomplete fixes.
• Write strong tests, reference patches, expected outcomes, and concise explanations of the failure mode.
• Review peer-authored tasks for realism, reproducibility, difficulty, and clean evaluation criteria.

What we look for:

• Strong practical depth in test strategy, automation frameworks, hidden and adversarial tests, reliability, and reproducible defect analysis.
• Ability to navigate unfamiliar code or specifications and communicate technical decisions in clear written English.
• Verifiable evidence such as meaningful GitHub work, shipped software, a competition profile, technical writing, or a portfolio. Confidential industry work can be described in the application without exposing proprietary material.
• Prior AI-evaluation experience is useful but not required; strong engineering judgement and testing discipline matter more.