Remote Python Engineer for Codebase Q&A AI Tasks

Weekday AI; United States; Remote; Part time

Job summary

Weekday AI is seeking experienced Python engineers to build the dataset that teaches AI agents to reason about real codebases.

You will build difficult codebase-exploration tasks: high-level engineering questions about production Python repositories that cannot be answered by reading a single file, and that require runtime evidence to resolve — the kind of question a senior engineer answers by actually investigating the system.

Qualifications

• 3+ years of professional software engineering experience with substantial production Python
• Ability to navigate a large unfamiliar Python codebase and explain runtime behavior
• Fluency with Python internals including import machinery, descriptors, metaclasses, async/event loops, packaging, and C-extensions

Responsibilities

• Select tasks from a pre-validated pool of Python repositories, each pinned to a specific git commit and with rubrics and a golden solution
• Explore the repository to understand the subsystem and its PR/issue history
• Rewrite the task question to challenge frontier coding agents while remaining well-defined
• Adjust rubrics and foils to reward correct reasoning and penalize plausible-but-wrong answers
• Run the Studio validation loop (Check, Golden, Run, Analyze) and iterate until all stages pass
• Document work in the task README and address reviewer feedback on returned tasks

Skills

Python

Codebase exploration

Runtime debugging

Git workflows

Communication

Tools

Okta

Insightful

Job description

Weekday AI is seeking experienced Python engineers to build the dataset that teaches AI agents to reason about real codebases.

You will build difficult codebase-exploration tasks: high-level engineering questions about production Python repositories that cannot be answered by reading a single file, and that require runtime evidence to resolve — the kind of question a senior engineer answers by actually investigating the system.