Frontend AI Code Evaluator - Remote Contractor

Talantir; France; Remote; EUR 55,000 - 96,000; Part time

Job summary

Talantir is seeking a remote Independent Contractor to rigorously evaluate AI-generated coding and software engineering responses. You will assess accuracy, reasoning, and explanations across multiple languages, contributing to high-quality AI coding outputs.

You will perform code execution tests, document findings, and provide reproducible evaluation artifacts. A BS/MS/PhD in CS, strong programming skills, and open-source contributions are required.

Qualifications

• BS, MS or PhD in Computer Science or closely related field.
• Significant real-world software engineering experience.
• Expertise in at least one language (Python, Java, C++, JavaScript, Go, Rust).
• Ability to solve Medium/Hard problems on HackerRank or LeetCode independently.
• Contributions to well-known open-source projects with merged PRs.
• Extensive experience using LLMs while coding and knowledge of strengths/weaknesses.
• Strong attention to detail and ability to evaluate complex reasoning.

Responsibilities

• Evaluate LLM-generated responses for coding and software queries for accuracy and clarity.
• Conduct fact-checking using trusted sources and references.
• Perform accuracy testing by executing code and validating outputs with appropriate tools.
• Annotate model responses noting strengths and areas for improvement.
• Assess code quality, readability, and algorithmic soundness.
• Ensure responses align with conversational guidelines and system policies.
• Apply evaluation standards using established taxonomies, benchmarks, and guidelines.

Skills

Computer Science degree

Software engineering experience

Python

Java

C++

JavaScript

LeetCode/HackerRank

Open source contributions

LLMs in coding

Attention to detail

Education

BS/MS/PhD in Computer Science

Job description

Talantir is seeking a remote Independent Contractor to rigorously evaluate AI-generated coding and software engineering responses. You will assess accuracy, reasoning, and explanations across multiple languages, contributing to high-quality AI coding outputs.

You will perform code execution tests, document findings, and provide reproducible evaluation artifacts. A BS/MS/PhD in CS, strong programming skills, and open-source contributions are required.