AI Task Designer for Software Engineers (Contract)

OpenTrain AI; Canada; Hybrid; CAD 135,000 - 231,000; Part time

Benefits offered by this job

Part-time engagement

Pay range: $70-$120 USD per hour

Location: Canada

Job summary

OpenTrain AI is hiring a Software Engineering AI Task Author to design and calibrate engineering challenges used to train and evaluate AI agents. You will create scenarios with integration failures, log-based debugging, and code-tracing tasks for real-world validation.

The role blends hands-on software engineering with evaluation design, requiring clear criteria and measurable outcomes to ensure tasks are authentic and appropriately challenging.

Qualifications

• Hands-on software engineering experience in industry or production environments.
• Ability to move from symptoms to root cause and implement fixes.
• Experience with logs, observability, and debugging.

Responsibilities

• Author task prompts based on realistic production-style failures and debugging workflows.
• Build task environments with codebases, logs, alerts, and system states that represent plausible engineering situations.
• Write scoring rubrics defining correct diagnoses and fixes, including validation criteria where applicable.
• Solve each task yourself, implement the fix, and verify that the expected outcome works.
• Adjust code complexity, log volume, failure modes, and ambiguity to calibrate task difficulty.
• Review and correct AI-drafted prompts or rubrics.

Skills

Software engineering

Debugging

Git/version control

Code review

CI/CD

Observability

Infrastructure experience

Job description

OpenTrain AI is hiring a Software Engineering AI Task Author to design and calibrate engineering challenges used to train and evaluate AI agents. You will create scenarios with integration failures, log-based debugging, and code-tracing tasks for real-world validation.

The role blends hands-on software engineering with evaluation design, requiring clear criteria and measurable outcomes to ensure tasks are authentic and appropriately challenging.