AI Evaluator - Python (Freelance Opportunity)

Biz Tech Consultants

New Delhi

On-site

INR 1,200,000 - 2,400,000

Full time

8 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Biz Tech Consultants seeks experienced software engineers to work on RLHF projects that support AI coding agent training and evaluation. You will design technical tasks, review AI-generated responses, and provide evidence-based feedback while ensuring task difficulty aligns with model outputs.

Collaboration with remote or international teams is common. Ideal candidates have 4–7 years building production systems, strong Javascript, Docker, and database experience, and a track record of measurable

Qualifications

  • 4–7 years of experience building production systems.
  • Strong hands-on skill in Javascript.
  • Experience with Docker and command line tools.
  • Experience with at least one relational and one NoSQL database.
  • Able to point to a clear before/after metric from past work.
  • Experience working with remote or international clients or teams.
  • Some exposure to secure data handling or compliance work.

Responsibilities

  • Design technical tasks across areas such as software development, debugging, data processing, security, and ML.
  • Set up environments and automated tests for task validation.
  • Review AI-generated responses for correctness and quality.
  • Provide clear, evidence-based written feedback and ratings.
  • Validate task or evaluation difficulty against AI model outputs and revise as needed.

Skills

Javascript
Remote collaboration
Data handling/compliance

Tools

Docker
Command line
Relational databases
NoSQL databases

Job description

Job Description : We are looking for experienced software engineers to work on RLHF projects that support the training and evaluation of AI coding agents. Depending on the project, the work can involve designing realistic technical tasks, or reviewing and rating AI generated responses for correctness, clarity, and reliability. It suits engineers who enjoy precise technical writing and careful quality review as much as building.

Key Responsibilities

  1. Design technical tasks across areas such as software development, debugging, data processing, security, and machine learning, and set up any supporting environment and automated tests needed to validate them, where the project calls for task authoring.
  2. Review AI generated responses to technical tasks, assessing correctness, code quality, and adherence to instructions.
  3. Provide clear, evidence based written feedback and comparative ratings across responses.
  4. Validate task or evaluation difficulty against AI model outputs and revise based on review feedback.

Desired Candidate ProfileHighly Required

  1. 4 to 7 years of experience building production systems, not just internal tools.
  2. Strong hands-on skill in Javascript
  3. Comfortable working with Docker and command line tools.
  4. Comfortable working with one relational and one NoSQL database.
  5. Can point to a clear before and after metric from past work.
  6. Experience working with remote or international clients or teams.
  7. Some exposure to secure data handling or compliance work.

Preferred

  1. Has integrated an AI or LLM API into a real, shipped feature.
  2. Experience in AI training and evaluation work.
  3. Prior experience debugging, reviewing code, and writing documentation or test cases.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Agent Evaluation Analyst (Freelance)
AI Agent Evaluation Analyst (Freelance)

Mindrift • Pune District

Remote
INR 1,858,273 - 2,106,043
Flexible remote work
Competitive pay up to $17/hour
Experience on advanced AI projects
AI Testing
AI Testing

Momento Cybertech Solutions • Hyderabad, Pune, Chennai

Hybrid
INR 1,200,000 - 2,000,000
AI Evaluator - Coding (LeetCode / DSA) | Freelance | Freshers
AI Evaluator - Coding (LeetCode / DSA) | Freelance | Freshers

Biz Tech Consultants • India

Remote
INR 482,000 - 827,000
AI Agent Evaluation Analyst (Freelance)
AI Agent Evaluation Analyst (Freelance)

Mindrift • Ahmedabad District

Remote
INR 1,237,589 - 1,762,627
Flexible project schedule
Competitive hourly rates
Experience in advanced AI projects
AI Agent Evaluation Analyst (Freelance)
AI Agent Evaluation Analyst (Freelance)

Mindrift • Maharashtra

Remote
INR 1,242,200 - 1,490,640
Flexible schedule
Competitive pay up to $12/hour
Gain experience in advanced AI projects
AI Engineer / AI Software Engineer-Task Creator (RL Environments) (Contract) Careerflow.ai ·
AI Engineer / AI Software Engineer-Task Creator (RL Environments) (Contract) Careerflow.ai ·

Dreamwork • India

Remote
INR 5,341,000 - 7,607,000
One free revision round
Flexible hours
Fully async collaboration
Remote AI Data Engineer
Remote AI Data Engineer

Turing • Chandigarh

Remote
INR 1,102,000 - 2,204,000
Aubergine Solutions - Senior AI Evaluation Andamp; Reliability Engineer
Aubergine Solutions - Senior AI Evaluation Andamp; Reliability Engineer

Aubergine Solutions • Ahmedabad

On-site
INR 2,800,000 - 4,500,000
AI Engineer (LLM & Agent Systems)
AI Engineer (LLM & Agent Systems)

ADscholars • Hyderabad

On-site
INR 1,500,000 - 2,700,000
Freelance AI Evaluation Scenario Writer
Freelance AI Evaluation Scenario Writer

Mindrift • New Delhi

Remote
INR 1,318,000 - 1,582,000
Flexible work schedule
Competitive pay up to $12/hour
Experience on advanced AI projects