Remote Browser Automation Quality Evaluator

AuraOne; United States; Remoto; USD 28.000 - 41.000; Part-time

Descrizione del lavoro

AI Trainer Jobs is seeking a remote Browser Automation and Tool-Use Specialist to evaluate model outputs against rubrics and provide structured feedback. You will compare paired responses, assign severity tags, and label content issues to support retraining efforts.

Prior evaluation experience and strong reasoning are essential for success. The role is a contractor position with hourly compensation, requiring reliable async availability of at least 10 hours per week and strict attention to

Competenze

• Prior evaluation, annotation, or human-rater experience on browser automation and tool use evaluation or adjacent content for Browser Automation and Tool-Use Specialist work.
• Comfort applying multi-page rubrics consistently across long batches.
• Clear written reasoning that names the issue and the rubric clause being applied.
• Strong attention to detail and the ability to flag when a prompt itself is the problem.
• Reliable async availability for at least 10 hours per week.

Mansioni

• Evaluate browser automation and tool use evaluation model outputs against a versioned rubric and assign severity tags for Browser Automation and Tool-Use Specialist assignments.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.

Conoscenze

Model evaluation

Rubric annotation

Severity tagging

Inter-rater calibration

Browser automation

AI evaluation

Rubric writing

Expert review

Descrizione del lavoro

AI Trainer Jobs is seeking a remote Browser Automation and Tool-Use Specialist to evaluate model outputs against rubrics and provide structured feedback. You will compare paired responses, assign severity tags, and label content issues to support retraining efforts.

Prior evaluation experience and strong reasoning are essential for success. The role is a contractor position with hourly compensation, requiring reliable async availability of at least 10 hours per week and strict attention to