Get a reply from this employer
Your resume and cover letter, tailored to exactly what they’re hiring for.
Parailabs is seeking a software engineering specialist to evaluate the quality and reproducibility of benchmark tasks used to train and evaluate frontier AI models. The role involves auditing reference patches, test runners, and Docker isolation, and delivering rubric-based feedback on repository-level tasks and grading integrity.
The ideal candidate has strong Python skills, open-source contributions, and experience auditing test harnesses and reward systems in AI benchmarks.
Evaluate the quality, correctness, and reproducibility of software-engineering benchmark tasks used to train and evaluate a frontier AI lab's models. You'll assess repository-level tasks, reference patches, test harnesses, and grading integrity — and provide clear, rubric-based written feedback.