AI Engineer / AI Software Engineer-Task Creator (RL Environments) (Contract) Careerflow.ai ·

Dreamwork

India

Remote

INR 5,341,000 - 7,607,000

Full time

5 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

One free revision round
Flexible hours
Fully async collaboration

Job summary

Careerflow.ai is hiring an AI Engineer to craft challenging RL-based task environments for testing AI agents on a Linux desktop.

You will design scenarios, assemble data, write agent instructions, and implement scoring in Python. This contract role is fully remote from India with a 3–4 week window and a per-task compensation of $50. Start next day after selection and collaborate asynchronously with a fast-moving team.

Qualifications

  • Proficient in Python scripting for real environments.
  • Background in AI/ML projects or evaluations.
  • Experience with RL environments or agent scoring.
  • Comfort with Ubuntu and Linux desktops.
  • Meticulous, detail-oriented to avoid flaky tasks.
  • Familiar with OSWorld or similar benchmarks is a plus.

Responsibilities

  • Design realistic multi-step tasks across several apps.
  • Assemble necessary deliverables: spreadsheets, docs, emails, data.
  • Write clear, giveaway-free instructions for the agent.
  • Develop Python to set up environments and scoring.
  • Test tasks, run them, and prepare concise docs.

Skills

Python
AI/ML background
RL environments
Ubuntu
VMs
Attention to detail

Job description

Home · Careerflow.ai · AI Engineer / AI Software Engineer-Task Creator (RL Environments) (Contract)

Careerflow Human Data Labs partners with AI companies to bring real-world professional expertise into their products.

About the work

We build hard, realistic tasks used to test how well AI agents do real work on a computer. Each task is a small scenario running on a Linux desktop — real files, real applications, real documents — plus a program that automatically checks whatever the agent produced.

You create those tasks end to end. The bar: hard for a top AI agent, straightforward for a competent human.

Responsibilities
  • Design a realistic multi-step scenario across a few applications.
  • Put together the files it needs — spreadsheets, documents, emails, data. Sometimes supplied to you, sometimes built by you.
  • Write the instruction the AI agent receives: clear, complete, no giveaways.
  • Write Python to set up the environment and to score the result automatically.
  • Test it, run it, and hand over short documentation.
What we are looking for
  • Python — comfortable writing and debugging real scripts.
  • Heavy AI user — you work with AI coding tools and models every day and get genuinely good output from them.
  • Some AI / ML background — worked on AI projects, at an AI company, or with agents and evaluations.
  • Ubuntu and virtual machines — basic comfort. Everything runs on hosted Linux VMs, so you should be fine on a terminal and inside a Linux desktop.
  • Careful and detailed — an unclear instruction or a sloppy check breaks the task.
  • You have built RL environments before — anything where an agent acts and a program scores the outcome.
  • You know OSWorld or similar computer-use benchmarks. Not required, but it means you will be productive on day one.
Payment and availability
  • $50 USD per task, paid once the task is accepted and approved in review.
  • One free revision round if a task needs fixes.
  • No cap on how many tasks you deliver — throughput is up to you.
  • Must be available to start the next day and stay through the 3–4 week window.
  • Flexible hours, fully async. We ask for a daily check-in and feedback turnaround within about a day.
Process

Short application → the screening questions below → a quick call → one paid pilot task ($50 on acceptance) → onboard. Decisions within 48 hours.

Engagement

Short-term contract, 3 to 4 weeks

Start

Immediately — the day after selection

Location

Fully remote, India Only

Hours

Flexible — we measure delivered tasks, not hours

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Software Engineer - Backend, Remote- Contract Remote (International)
Software Engineer - Backend, Remote- Contract Remote (International)

Xperteez Technology Pvt Ltd • India

Remote
INR 827,000 - 1,653,000
Senior Software Engineer (Remote | $50 –$100/hr)
Senior Software Engineer (Remote | $50 –$100/hr)

Synthires • India

On-site
INR 31,573,000 - 36,083,000
Flexible schedule
Part-time hours available
AI Engineer (Remote, Full-Time) [HR163]
AI Engineer (Remote, Full-Time) [HR163]

Smart Working • Delhi

Remote
INR 1,200,000 - 1,800,000
Fixed Shifts
No Weekend Work
Day 1 Benefits: Laptop and full medical insurance
+2
AI Engineer (Remote, Full-Time) [HR163]
AI Engineer (Remote, Full-Time) [HR163]

Smart Working • Ernakulam

Remote
INR 1,000,000 - 1,500,000
Fixed shifts
No weekend work
Laptop and full medical insurance
+2
Software Engineer, AI Coding Tasks (Contract)
Software Engineer, AI Coding Tasks (Contract)

Askable • Gopalganj

Hybrid
INR 3,828,000 - 4,774,000
AI Engineer (Remote, Full-Time) [HR163]
AI Engineer (Remote, Full-Time) [HR163]

Smart Working • Chennai District

Remote
INR 1,200,000 - 1,500,000
Fixed Shifts: 12:00 PM - 9:30 PM IST
No Weekend Work
Day 1 Benefits: Laptop and full medical insurance
+2
AI Engineer (Python, LLMs)
AI Engineer (Python, LLMs)

Nihar • India

Remote
INR 1,000,000 - 1,500,000
Evaluation Scenario Writer - AI Agent Testing Specialist
Evaluation Scenario Writer - AI Agent Testing Specialist

Mindrift • Hyderabad

Remote
INR 1,236,467 - 1,761,029
Software Engineer (Remote | $60–$120/hr)
Software Engineer (Remote | $60–$120/hr)

Synthires • Bengaluru

Remote
INR 7,761,000 - 15,522,000
Fully remote worldwide
Flexible schedule
15+ hours per week
AI Evaluator - Python (Freelance Opportunity)
AI Evaluator - Python (Freelance Opportunity)

Biz Tech Consultants • New Delhi

On-site
INR 1,200,000 - 2,400,000