Algorithmic Problem Setter & Test-Case Architect

Socket.dev

Manitoba

Hybrid

CAD 57,000 - 172,000

Full time

7 days ago
Be an early applicant

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

Mindrift seeks contributors to craft competitive programming problems and their automatic graders for AI model training. You will design original problems, provide correct and deliberately-wrong solutions, and build robust test cases to ensure reliable signals during evaluation.

The role emphasizes strong C++ and Python skills, a solid competitive programming background, and excellent written English. Projects are time-bound and pay per task, with compensation up to $90 per hour based on

Qualifications

  • Strong competitive programming background, shown by public profile or IOI/ICPC/national-olympiad record.
  • Fluent contest C++ (STL, complexity); comfortable writing Python.
  • Can explain why a solution is wrong and build an input that breaks it.

Responsibilities

  • Write original algorithmic problems — clear statement, sound constraints, an intended solution.
  • Write correct solutions in C++ and Python.
  • Write realistic wrong solutions (common mistakes) to test against.
  • Build and harden test cases: generators, edge cases, stress and "hacking" tests.
  • Write checkers/interactors (testlib) for problems with more than one valid answer.
  • Confirm the quality bar: correct solutions pass, wrong ones fail, within time limits.

Skills

Competitive programming
C++
Python
Written English (C1+)

Tools

Testlib

Job description

Mindrift seeks contributors to craft competitive programming problems and their automatic graders for AI model training. You will design original problems, provide correct and deliberately-wrong solutions, and build robust test cases to ensure reliable signals during evaluation.

The role emphasizes strong C++ and Python skills, a solid competitive programming background, and excellent written English. Projects are time-bound and pay per task, with compensation up to $90 per hour based on

Get your free, confidential resume review.
or drag and drop your file here.
Similar jobs

Similar jobs worth comparing

Freelance AI Task Architect & Backend Engineer
Freelance AI Task Architect & Backend Engineer

Dorado • Canada

Hybrid
CAD 34,000 - 48,000
Software Engineering Evaluation Specialist
Software Engineering Evaluation Specialist

Socket.dev • Quebec

On-site
CAD 48,000 - 67,000
Freelance Software Engineer - AI Coding Agent Evaluation Mindrift · Remote Pay not listed →
Freelance Software Engineer - AI Coding Agent Evaluation Mindrift · Remote Pay not listed →

Dorado • Canada

Hybrid
CAD 34,000 - 48,000
Freelance Machine Learning Engineer (Python)
Freelance Machine Learning Engineer (Python)

Mindrift • Canada

Remote
Competitive hourly rates
Participation in advanced AI projects
Flexible working hours
Freelance Software Developer (Ruby) - AI Trainer
Freelance Software Developer (Ruby) - AI Trainer

Mindrift • Canada

Remote
Competitive pay rates up to $48/hour
Flexibility to work around other commitments
Experience on advanced AI projects
Freelance Automotive Engineering & Python Expert - AI Trainer
Freelance Automotive Engineering & Python Expert - AI Trainer

Mindrift • Canada

Remote
Freelance Automotive Engineering & Python Expert - AI Trainer
Freelance Automotive Engineering & Python Expert - AI Trainer

Mindrift • Canada

On-site
Competitive pay up to $38/hour
Flexible hours
Freelance Software Developer (Ruby) - AI Trainer
Freelance Software Developer (Ruby) - AI Trainer

Mindrift • Canada

Remote
Competitive hourly pay up to $45/hour
Part-time, remote project
Experience with advanced AI projects
Freelance Machine Learning Engineer (Python)
Freelance Machine Learning Engineer (Python)

Mindrift • Canada

Remote
Competitive pay up to $55/hour
Flexible remote work schedules
Gain experience on advanced AI projects
Freelance AI Evaluation Scenario Writer
Freelance AI Evaluation Scenario Writer

Mindrift • Canada

Remote
Flexible work schedule
Competitive pay up to $45/hour
Opportunity to enhance your portfolio
+1