Remote AI Task Engineer for Real-World Agent Scenarios

S27a; United States; Remote; USD 15,000 - 35,000; Full time

Benefits offered by this job

Flexible hours

Fully remote

Paid pilot task

Job summary

Careerflow Human Data Labs seeks a detail-oriented contractor to design realistic, multi-step scenarios for AI evaluation. You will assemble documents and data, provide precise instructions to AI agents, and write Python scripts to automate environment setup and scoring. Strong Linux/VM experience and Git basics are essential.

Work is fully remote with a 3–4 week engagement, starting immediately. The role offers flexible hours and a quick onboarding process, with paid pilot tasks and ongoing

Qualifications

• Proficient in writing Python scripts to set up test environments and scoring pipelines.
• Experience using AI tools and models to develop evaluation tasks.
• Familiarity with AI/ML projects and agent-based benchmarks preferred.
• Comfortable working on Linux desktops and hosted VMs.
• Strong attention to detail to ensure clear, verifiable tasks.

Responsibilities

• Design realistic, multi-step scenarios across several applications.
• Assemble necessary files: spreadsheets, documents, emails, data.
• Write clear instructions for AI agents with no giveaways.
• Develop Python scripts to automate setup and scoring of tasks.
• Test tasks, run experiments, and document the process.

Skills

Python

Git

OS/Linux terminal

Attention to detail

AI tooling familiarity

Tools

Ubuntu

Virtual machines

Job description

Careerflow Human Data Labs seeks a detail-oriented contractor to design realistic, multi-step scenarios for AI evaluation. You will assemble documents and data, provide precise instructions to AI agents, and write Python scripts to automate environment setup and scoring. Strong Linux/VM experience and Git basics are essential.

Work is fully remote with a 3–4 week engagement, starting immediately. The role offers flexible hours and a quick onboarding process, with paid pilot tasks and ongoing