Python Codebase Architect for AI Reasoning Tasks

Obsidian; Miami (FL); On-site; USD 148,000 - 210,000; Full time

Job summary

Mercor is seeking experienced Python engineers to build a dataset that teaches AI agents to reason about real codebases. You will tackle tasks across Django, Twisted, Scrapy, SymPy, and more, evaluating production Python repositories with runtime evidence and rigorous rubrics.

Fully remote, contract-based, paid per approved task, with tasks selected from a pool and evaluated through a multi-stage validation process. You’ll work asynchronously with global team leads from multiple time zones.

Qualifications

• 3+ years of professional software engineering experience, with substantial production Python

Responsibilities

• Select tasks from a pre-validated pool of Python repositories, each pinned to a specific git commit and arriving with an engineering question, positive and negative rubrics, foils, and a golden solution
• Explore the repository until you genuinely understand the subsystem in question, including the relevant PR or issue history
• Rewrite the task's question so it defeats frontier coding agents while remaining well-defined and fairly answerable
• Adjust rubrics and foils so they reward correct reasoning and reject plausible-but-wrong answers
• Run the full validation loop in Studio — Check, Golden, Run, Analyze — and iterate until every stage passes
• Document your work in the task README and submit for expert review, addressing reviewer feedback on returned tasks

Skills

Python

Codebase exploration

Runtime analysis

Git & CLI

Tools

Git

Containerization

Job description

Mercor is seeking experienced Python engineers to build a dataset that teaches AI agents to reason about real codebases. You will tackle tasks across Django, Twisted, Scrapy, SymPy, and more, evaluating production Python repositories with runtime evidence and rigorous rubrics.

Fully remote, contract-based, paid per approved task, with tasks selected from a pool and evaluated through a multi-stage validation process. You’ll work asynchronously with global team leads from multiple time zones.