AI Evaluation Specialist - RLHF, SFT & MCP (Remote, Immediate Joiners)

OpsLabel; India; Remoto; INR 1.653.000 - 3.306.000; Tempo pieno

Descrizione del lavoro

OpsLabel is expanding its expert network and seeks AI evaluation specialists for RLHF, SFT, and MCP evaluation. You will judge AI outputs, stress test reasoning, and tool use against detailed rubrics on frontier models.

Role is remote with India preference, freelance/contract for long-term projects. Strong technical minds in ML, software, linguistics, or STEM welcome; advanced degree a plus but not required.

Competenze

• Strong judgment on the quality, correctness, and safety of AI model outputs.
• Comfort following detailed rubrics and guidelines.
• Technical or analytical background in ML, software, data, linguistics, or STEM.

Mansioni

• Rank and compare model responses to build high-quality preference data.
• Write and review instruction-response and chain-of-thought examples.
• Evaluate agent tool-use: tool-calling accuracy and multi-step workflows.
• Catch subtle errors and reasoning failures.
• Deliver high-quality work across multiple sprints with feedback from senior reviewers.

Conoscenze

Strong judgment

Technical/analytical background

Prompt engineering

Strumenti

RLHF

SFT

MCP

Tool calling

Descrizione del lavoro

AI Evaluation Specialist - RLHF, SFT & MCP (Remote, Immediate Joiners)

OpsLabel is expanding its expert network, and we have an immediate need for AI evaluation specialists. We need your judgment on frontier AI projects spanning RLHF, SFT, and MCP (agent tool-use) evaluation, helping train and stress-test how well models respond, reason, and use tools.

• Remote (India preferred)
• Freelance / Contract, long-term project work
• Open to strong technical minds: ML, engineering, linguistics, or a STEM domain.
• Advanced degree a plus, not required. What matters is sound judgment on what a good AI answer looks like.

Must have:

• Strong judgment on the quality, correctness, and safety of AI model outputs
• Comfort working to detailed rubrics and guidelines
• A technical or analytical background (ML, software, data, linguistics, or a STEM field)

Nice to have:

• Hands‑on experience with RLHF, preference ranking, or reward modeling
• SFT data creation: instruction-response pairs or chain‑of‑thought
• Familiarity with MCP, function or tool calling, or agent workflows
• Programming or prompt‑engineering background

What you’ll do:

• Rank and compare model responses to build high‑quality preference data
• Write and review instruction‑response and chain‑of‑thought examples
• Evaluate agent tool‑use: tool‑calling accuracy and multi‑step workflows
• Catch the subtle errors and reasoning failures generic reviewers miss
• Deliver high‑quality work across multiple sprints with direct feedback from senior reviewers

Why this role matters:

• Your judgment directly shapes how the next generation of AI responds, reasons, and uses tools. This is high‑impact, flexible work for experts who want their depth to count, not faceless annotation work.Engagement model:Flexible hours, work on your own schedule
• Long‑term engagements on frontier AI projects
• Multiple delivery sprints across 3 to 6+ months