Onsite AI Data Quality & Evaluation Specialist

Welo Global; Sunnyvale (CA); On-site; USD 39,000 - 55,000; Full time

Benefits offered by this job

Paid Vacation

Health Insurance

401(k) Retirement Plan

Free Gourmet Food

Commuter Benefits

Job summary

Welocalize is seeking a Data Labeling Associate to evaluate AI model outputs and provide structured feedback at a fast pace. You will audit data and model behavior, identify gaps, and help refine evaluation guidelines for real-world applicability.

The role emphasizes critical thinking, attention to detail, and the ability to work with evolving guidelines in a collaborative, onsite environment.

Qualifications

• Native-level Marathi language proficiency and a university degree.
• B2 or superior level of English.
• 1–2 years of professional writing experience with structured writing skills.
• 2+ years of relevant experience in safety considerations for GenAI data delivery.
• Strong critical thinking and attention to detail.
• Ability to make sound judgment calls in ambiguous situations.
• Naturally curious about AI, technology, and how systems behave.

Responsibilities

• Evaluate AI model outputs and provide structured, high-quality feedback.
• Perform audit-based reviews of data and model behavior—identifying patterns, edge cases, and failure modes.
• Apply guidelines thoughtfully and flag when they don’t reflect real-world scenarios.
• Contribute to improving evaluation frameworks, not just executing them.
• Identify trends in model performance and communicate insights clearly.
• Participate in team discussions, calibrations, and stakeholder syncs.
• Partner with leads and cross-functional teams to refine quality standards.
• Document findings in a clear, concise, and actionable way.

Skills

Marathi language

English proficiency

Critical thinking

Attention to detail

Communication skills

Curiosity about AI

Data quality / QA

Judgment in ambiguity

Education

University degree

Job description

Welocalize is seeking a Data Labeling Associate to evaluate AI model outputs and provide structured feedback at a fast pace. You will audit data and model behavior, identify gaps, and help refine evaluation guidelines for real-world applicability.

The role emphasizes critical thinking, attention to detail, and the ability to work with evolving guidelines in a collaborative, onsite environment.