RLHF Quality Analyst

Desicrew Solutions; Chennai District; On-site; INR 300,000 - 450,000; Full time

Job summary

DesiCrew is seeking a detail-oriented reviewer to judge language model outputs. You will rank responses for helpfulness, accuracy, and safety, applying safety policies to borderline content and recording the rationale behind every decision.

Work is on-site at the delivery centre in Chennai, with training provided. Suitable for early-career candidates with a bachelor’s degree or equivalent experience; additional experience in content moderation or linguistics is a plus.

Qualifications

• Strong reading comprehension and written English.
• Attention to detail and the patience for repetitive, judgement-heavy work.
• Willingness to have your judgement measured, discussed and corrected in calibration.
• Bachelor's degree or equivalent experience; training provided.
• Helpful, not required: content moderation, editorial, teaching, linguistics or translation experience, or a second Indian language.

Responsibilities

• Review and rank model responses for helpfulness, accuracy and safety.
• Apply trust safety policy to borderline content and record reasoning, not just the call.
• Flag rubric gaps and recurring ambiguity for guideline fixes.
• Participate in calibration sessions and golden-set checks for consistency.
• Maintain the quality bar under deadline without drifting from the rubric.

Skills

Reading comprehension

Written English

Attention to detail

Calibration

Team collaboration

Education

Bachelor's degree or equivalent

Job description

About the role

You will judge what a language model produces - ranking responses, applying safety policy and writing the preference data that model training runs on. The work is careful reading and consistent judgement, at volume, against a rubric that keeps moving as the policy sharpens.

What you will do

• Review and rank model responses for helpfulness, accuracy and safety against a written rubric.
• Apply trust safety policy to borderline content and record why a call was made, not only what it was.
• Flag rubric gaps and recurring ambiguity so the guideline gets fixed rather than worked around.
• Take part in calibration sessions and golden-set checks that keep judgement consistent across the team.
• Maintain the quality bar under deadline without drifting from the rubric.

What you will bring

• Strong reading comprehension and written English - this is the core skill of the role.
• Evident care with detail and the patience for repetitive, judgement-heavy work.
• Willingness to have your judgement measured, discussed and corrected in calibration.
• A bachelors degree in any discipline, or equivalent experience. This role is open to candidates early in their career; the training is provided.
• Helpful, not required: content moderation, editorial, teaching, linguistics or translation experience, or a second Indian language.

Working arrangements

Full time, on site at the delivery centre listed above. Standard working hours, with shift cover arranged within the team where a client programme runs to a different timezone.

About DesiCrew

DesiCrew has run high-stakes AI and enterprise operations from small-town and rural India since 2007. More than 2,000 people work across our delivery centres, 68% of them women - roughly twice the industry average - and 49% the sole earner in their family. We are ISO 9001, ISO 27001 and ISO 42001 certified and a Great Place to Work.

We hire for aptitude and train for the rest. DTOUCH, our in-house learning platform, has delivered more than 5,000 certifications and 1,200 AI-certified professionals, and training continues well past onboarding.