QA Software Engineer - Code Review Expert

Talantir; Francia; A distancia; EUR 68.000 - 114.000; A tiempo parcial

Descripción de la vacante

Talantir is partnering with a top AI research organization to evaluate and improve how coding assistants reason, act, and communicate during development workflows. We’re seeking technically sharp experts to assess full transcripts of user–AI coding conversations on a short-term basis.

Ideal candidates include senior engineers with code-review/execution insight, QA specialists, or technical writers. The role is fully remote with weekly payments and the potential for recurring batches.

Formación

• Senior/Staff engineers with code review and execution insight.
• QA engineers with verification habits.
• Technical writers skilled at comparing instructions vs. implementation.

Responsabilidades

• Review long-form transcripts between users and AI coding assistants.
• Analyze the AI’s logic, execution, and stated actions.
• Score transcripts using a 10-point rubric across multiple criteria.
• Optionally write brief justifications citing examples from the dialogue.
• Detect mismatches between claims and actions.

Conocimientos

Code review

Execution insight

QA verification

Documentation

Python

Git workflows

Testing frameworks

Debugging tools

APIs

Herramientas

Git

Testing frameworks

Debugging tools

APIs

Descripción del empleo

Partnering with a top AI research organization to evaluate and improve how coding assistants reason, act, and communicate during development workflows. We’re seeking technically sharp experts (especially those with experience in code review, testing, or documentation) to assess full transcripts of user–AI coding conversations. This short-term engagement helps shape the future of developer-assisting AI systems.

Key Responsibilities

• Review long-form transcripts between users and AI coding assistants
• Analyze the AI’s logic, execution, and stated actions in detail
• Score each transcript using a 10-point rubric across multiple criteria
• Optionally write brief justifications citing examples from the dialogue
• Detect mismatches between claims and actions (e.g., saying "I’ll run tests" but not doing so)

Ideal Qualifications

Top choices:

• Senior or Staff Engineers with deep code review experience and execution insight
• QA Engineers with strong verification and consistency-checking habits
• Technical Writers or Documentation Specialists skilled at comparing instructions vs. implementation

Also a strong fit:

• Backend or Full-Stack Developers comfortable with function calls, APIs, and test workflows
• DevOps or SRE professionals familiar with tool orchestration and system behavior analysis

Languages and Tools:

• Proficiency in Python is helpful (most transcripts are Python-based)
• Familiarity with other languages like JavaScript, TypeScript, Java, C++, Go, Ruby, Rust, or Bash is a plus
• Comfort with Git workflows, testing frameworks, and debugging tools is valuable

More About the Opportunity

• Must complete each transcript batch within 5 hours of starting (unlimited tasks to be done)
• Flexible, task-based engagement with potential for recurring batches

We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.

Contract and Payment Terms

• You will be engaged as an independent contractor.
• This is a fully remote role that can be completed on your own schedule.
• Projects can be extended, shortened, or concluded early depending on needs and performance.
• Your work at will not involve access to confidential or proprietary information from any employer, client, or institution.
• Payments are weekly on Stripe or Wise based on services rendered.