A complete application in a minute
Tailored resume and cover letter, ready to send.
Marketframe is seeking a software engineer to evaluate the quality, correctness, and reproducibility of benchmark tasks used for frontier AI model training and evaluation. You will assess repository-level tasks, reference patches, test harnesses, and grading integrity, providing rubric-based written feedback.
The role requires 3+ years of professional software engineering experience, strong open-source involvement, and proficiency in Python plus at least one of Java, Go, TypeScript, or C++.
Evaluate the quality, correctness, and reproducibility of software-engineering benchmark tasks used to train and evaluate a frontier AI lab's models. You'll assess repository-level tasks, reference patches, test harnesses, and grading integrity — and provide clear, rubric-based written feedback.