Software Engineer Task Author

Alignerr

United States

Remote

USD 120,000 - 160,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Alignerr is building a dataset of expert tasks to train and evaluate AI agents on real enterprise engineering work. As a Task Author for the Software Engineer role, you will design engineering tasks that diagnose failures, trace bugs, and write and verify fixes with hands‑on coding work.

You will write precise scoring rubrics, set up realistic task environments, and validate tasks by solving them yourself to ensure the rubric and rubric‑driven checks are sound.

Qualifications

  • Working software engineer with hands‑on industry experience.
  • Fluency in Git, version control workflows, and code review practices.
  • Strong debugging, observability, and log analysis skills.
  • Experience with infrastructure, integrations, and configuration code (APIs, services, CI/CD).
  • Ability to write and verify a code fix in a live or simulated environment.
  • Ability to define precise, checkable correctness criteria for engineering outcomes.

Responsibilities

  • Author realistic engineering task prompts: integration failures, log‑based debugging, bug traces through source code, and configuration or integration fixes.
  • Write scoring rubrics that define exactly what a correct fix or diagnosis looks like and how it is verified.
  • Set up task environments with realistic codebases, logs, alerts, and system states that drop an AI agent into a plausible engineering situation.
  • Solve each task yourself — write the actual fix and confirm it works — to validate the task is sound and the rubric is accurate.
  • Calibrate task difficulty — adjust code complexity, log volume, failure modes, or ambiguity until the task reliably challenges the model to the intended degree.
  • Review and correct AI‑drafted task prompts or rubrics when provided.

Skills

Hands-on engineering
Git / version control
Debugging skills
Observability tools
Code tracing

Tools

CI/CD
Datadog
Grafana

Job description

About the Role

Alignerr is building a dataset of expert tasks that train and evaluate advanced AI agents on real enterprise work. As a Task Author for the Software Engineer role, you will design and calibrate real engineering tasks — diagnosing failing integrations from logs, tracing bugs through a codebase, and writing and verifying fixes — that genuinely challenge a capable AI agent. This role requires hands‑on engineering skill; generalist profiles cannot do this work.

Key Responsibilities
  • Author realistic engineering task prompts: integration failures, log‑based debugging, bug traces through source code, and configuration or integration fixes.
  • Write scoring rubrics that define exactly what a correct fix or diagnosis looks like and how it is verified — including live environment validation where applicable.
  • Set up task environments with realistic codebases, logs, alerts, and system states that drop an AI agent into a plausible engineering situation.
  • Solve each task yourself — write the actual fix and confirm it works — to validate the task is sound and the rubric is accurate.
  • Calibrate task difficulty — adjust code complexity, log volume, failure modes, or ambiguity until the task reliably challenges the model to the intended degree.
  • Review and correct AI‑drafted task prompts or rubrics when provided.
Qualifications
  • Working software engineer with hands‑on experience in industry — this is not a theoretical or instructional role.
  • Fluency in Git, version control workflows, and code review practices.
  • Strong debugging skills: comfortable reading logs, using observability tools, and tracing failures through a codebase.
  • Experience with infrastructure, integrations, and configuration code (APIs, services, config files, CI/CD).
  • Ability to write and verify a code fix in a live or simulated environment.
  • Ability to define precise, checkable correctness criteria for engineering outcomes.
Nice to Have
  • Experience in manufacturing, industrial IoT, or enterprise SaaS engineering contexts.
  • Background with monitoring/observability platforms (e.g., Datadog, PagerDuty, Grafana).
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Software Engineer - AI Task Designer and Debugging Expert
Software Engineer - AI Task Designer and Debugging Expert

Alignerr • United States

Remote
USD 120,000 - 160,000
Customer Support / Success Task Author
Customer Support / Success Task Author

Alignerr • United States

Remote
USD 70,000 - 110,000
Sales / Revenue Operations Task Author
Sales / Revenue Operations Task Author

Alignerr • United States

Remote
USD 90,000 - 130,000
Task Designer, Software Engineering
Task Designer, Software Engineering

Reinforced Learning • Town of Tonawanda (NY)

On-site
USD 95,000 - 140,000
Finance / Accounting Task Author
Finance / Accounting Task Author

Alignerr • United States

Remote
USD 70,000 - 100,000
Marketing Task Author
Marketing Task Author

Alignerr • United States

Remote
USD 60,000 - 90,000
Sales & Revenue Operations Task Author (AI Training)
Sales & Revenue Operations Task Author (AI Training)

Alignerr Corp. • United States

Remote
USD 33,000 - 67,000
Software Engineer
Software Engineer

Remotebridge • United States

Remote
USD 6,000 - 10,000
Expert: Software Engineer – AI Evaluation (Task-based Contract, Remote)
Expert: Software Engineer – AI Evaluation (Task-based Contract, Remote)

Patronus AI • United States

Remote
USD 69,000 - 138,000
Flexible schedule
Remote contract
15–20 hours per week
Technical Problem Author (Multi-Domain)
Technical Problem Author (Multi-Domain)

Auraone • United States

Remote
USD 99,000 - 135,000