AI/ML Evaluator - Human-in-the-Loop (Freelance)

Welocalize; Town of Texas (WI); On-site; USD 192,864,000 - 220,416,000; Full time

Job summary

Welocalize is seeking analytical and technically skilled AI/ML Evaluators to review and evaluate complex AI system behaviour using expert human judgment. You will analyse AI system outputs, telemetry and other technical signals to identify meaningful patterns and distinguish legitimate user activity from automated or bot activity.

You'll provide high-quality human evaluations to train, evaluate and improve AI systems, applying your knowledge of AI/ML concepts and detailed project guidelines with

Qualifications

• Educational background or equivalent experience in AI, ML, CS, data science, statistics, engineering, or related field.
• Experience with AI/ML systems, data analysis, or technical system analysis.
• Strong understanding of core AI/ML concepts.
• Excellent written English and documentation skills.
• Familiarity with bot detection, anomaly detection, or fraud detection is preferred.
• Experience with data annotation or model evaluation is preferred.

Responsibilities

• Review and evaluate AI system behaviour, outputs, and technical data according to project guidelines.
• Analyse system telemetry, event data, signals, and other technical information to identify meaningful patterns.
• Evaluate patterns and signals that distinguish legitimate user activity from automated or bot activity.
• Apply AI/ML knowledge and expert human judgment when reviewing complex or ambiguous cases.
• Identify unusual patterns, inconsistencies, or behaviours that may require further review.
• Evaluate complex cases using available evidence, context, and defined project requirements.
• Classify, label, or annotate assigned data accurately and consistently.
• Provide high-quality human evaluations to support AI system training, evaluation, and improvement.
• Document decisions and supporting reasoning clearly when required.
• Identify unclear or unusual cases and flag them according to defined project processes.
• Apply detailed project guidelines consistently across assigned tasks.
• Maintain high levels of quality, accuracy, consistency, and attention to detail.

Skills

Analytical thinking

AI/ML concepts

Data analysis

Pattern recognition

Documentation skills

Education

Bachelor's degree or equivalent in AI/ML/CS/Data Science/Statistics/Engineering

Job description

Welocalize is seeking analytical and technically skilled AI/ML Evaluators to review and evaluate complex AI system behaviour using expert human judgment. You will analyse AI system outputs, telemetry and other technical signals to identify meaningful patterns and distinguish legitimate user activity from automated or bot activity.

You'll provide high-quality human evaluations to train, evaluate and improve AI systems, applying your knowledge of AI/ML concepts and detailed project guidelines with