AI Evaluator - Freelance Opportunity (Python)

Biz Tech Analytics; New Delhi; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

Biz Tech Analytics is seeking experienced software engineers to work on RLHF projects supporting AI coding agents. You will design technical tasks and validate them with automated tests, or review AI-generated responses for accuracy and clarity.

The role emphasizes precise technical writing, high-quality reviews, and collaboration with international clients. Remote-friendly freelance engagement is possible, with emphasis on building production-grade systems.

Qualifications

• 4 to 7 years of experience building production systems, not just internal tools.
• Strong hands-on skill in Javascript.
• Comfortable working with Docker and command line tools.
• Comfortable working with one relational and one NoSQL database.
• Can point to a clear before and after metric from past work.
• Experience working with remote or international clients or teams.
• Some exposure to secure data handling or compliance work.

Responsibilities

• Design technical tasks across areas such as software development, debugging, data processing, security, and machine learning, and set up any supporting environment and automated tests needed to validate them, where the project calls for task authoring.
• Review AI generated responses to technical tasks, assessing correctness, code quality, and adherence to instructions.
• Provide clear, evidence based written feedback and comparative ratings across responses.
• Validate task or evaluation difficulty against AI model outputs and revise based on review feedback.

Skills

JavaScript

Node.js

Go

Rust

Python

Docker

Git

CI/CD

System Design

LLM Integration

NoSQL

Kubernetes

Kafka

RabbitMQ

gRPC

Redis

Tools

Docker

Kubernetes

PostgreSQL

MongoDB

Git

pytest

CI/CD

gRPC

Redis

LLM API integration

Job description

Job Description

We are looking for experienced software engineers to work on RLHF projects that support the training and evaluation of AI coding agents. Depending on the project, the work can involve designing realistic technical tasks, or reviewing and rating AI generated responses for correctness, clarity, and reliability. It suits engineers who enjoy precise technical writing and careful quality review as much as building.

Key Responsibilities

1. Design technical tasks across areas such as software development, debugging, data processing, security, and machine learning, and set up any supporting environment and automated tests needed to validate them, where the project calls for task authoring.
2. Review AI generated responses to technical tasks, assessing correctness, code quality, and adherence to instructions.
3. Provide clear, evidence based written feedback and comparative ratings across responses.
4. Validate task or evaluation difficulty against AI model outputs and revise based on review feedback.

Desired Candidate ProfileHighly Required

1. 4 to 7 years of experience building production systems, not just internal tools.
2. Strong hands-on skill in Javascript
3. Comfortable working with Docker and command line tools.
4. Comfortable working with one relational and one NoSQL database.
5. Can point to a clear before and after metric from past work.
6. Experience working with remote or international clients or teams.
7. Some exposure to secure data handling or compliance work.

Preferred

1. Has integrated an AI or LLM API into a real, shipped feature.
2. Experience in AI training and evaluation work.
3. Prior experience debugging, reviewing code, and writing documentation or test cases.

Employment Type Contractual/FreelanceKey Skills

Python, Java, JavaScript, Node.js, Go, Rust, Docker, Git, pytest, AI Integration, LLM, NoSQL, CI/CD, System DesignNice to Have Kubernetes, Kafka, RabbitMQ, gRPC, Redis