Top GitHub Contributors – Agentic Review Project

AI Work Expert; San Francisco (CA); Hybrid; USD 83,000 - 165,000; Full time

Job summary

AI Work Expert is seeking a contractor to design and develop real-world problems for an Agentic development environment. You will implement tasks in Python, create environments, and provide a golden solution and test notebooks to evaluate model performance.

The role emphasizes strong expertise in data science, ML, coding, or frontier STEM, with a demand for reliable weekly engagement (30+ hours). Remote work is supported, and the project is focused on advancing AI evaluation tasks.

Responsibilities

• Design challenging, real-world problems to target core capability loss failures identified in a leading model.
• Content Generation: Integrate the problems into an Agentic development environment using Python, with instructions, environment, and metadata.
• A Golden solution that follows the instructions.
• Evaluation and Analysis: Evaluate cross-model performance on the tasks.
• Headroom Identification: Identify tasks where target model fails to pass all tests, classifying failures as logical reasoning.

Job description

# Top GitHub Contributors – Agentic Review Projectper hourContractor Remote Last verified 10 Nov 2026Listed on Turing turing.comApplyApplyReferral link. You pay nothing; we may earn a fee.Affiliate disclosureWork Expert (WE) is an independent publishing and referral website. We are not a recruiter, hiring manager, agent or employer, and we are not affiliated with or endorsed by Turing. Applying takes you to the platform's own website, where we may be recorded as the referring source. We may receive a referral fee at no additional cost to you. Read our full affiliate disclosure →Apply within 30 days of clicking - Turing only credits applications inside its 30-day attribution window.## About the RoleAbout Us Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems.## What You'll Do* Task Design and Development: Design challenging, real-world problems that serve as the foundation for a model. Problems should be constructed to target specific core capability loss failures identified in a leading model.* Content Generation: Integrate the problems into an Agentic development environment, preparing all necessary components using Python, which include: Detailed Instructions and an overview of the required task.* A Golden solution that follows the instructions.* The necessary Environment, including datasets, Python libraries, and metadata.* A Test notebook containing unit tests that solutions must pass.* Evaluation and Analysis: Evaluate the cross model’s performance on the tasks.* Headroom Identification: Identify tasks where target model fails to pass all tests, specifically classifying the failure as a logical reasoning failure.* Loss Extraction: Analyze the agent’s steps (Agent Trajectory) to observe and extract core capability loss patterns from the model.## You're a Good Fit If You* Required skills: rust* Highly-skilled GitHub contributors.* Applicants must have strong expertise in either data science, ML, coding, or a deep quantitative background in frontier STEM.* Able to engage reliably for 30+ hrs per week (on weekdays).Get ready for TuringDesign a resume for Turing Prepare for the Turing interview## Role HighlightsPayPay not listedContractContractorLocationRemoteCategoryModel evaluationPosted8 months agoJob IDWE-8428## Pay & PayoutIndependent contractor engagement through Turing. Apply via the link below - Turing handles vetting, onboarding and payment.