RAG Source Relevance Analyst

Globik AI; United States; Remote; Full time

Job summary

Globik AI is seeking a RAG Source Relevance Specialist to evaluate whether documents, passages, or knowledge-base entries retrieved by an AI system are relevant to a user’s question. You will review queries and retrieved sources, determine usefulness, and flag irrelevant, outdated, or missing information to improve results.

Responsibilities include assessing content relevance, matching passages to reference documents, and documenting ambiguous or weak sources while providing structured feedback

Responsibilities

• Review user queries together with retrieved documents or passages.
• Determine whether retrieved sources are relevant to the query.
• Assess whether retrieved content contains information needed to answer the question.
• Identify irrelevant, redundant, outdated, or weak source material.
• Identify missing information needed to support a correct answer.
• Compare retrieved passages with reference documents when provided.
• Evaluate whether retrieval provides sufficient context for generation.
• Categorize retrieval errors using project frameworks.
• Document ambiguous retrieval cases.
• Provide structured feedback to improve retrieval quality.

Job description

Globik AI is seeking a RAG Source Relevance Specialist to evaluate whether documents, passages, or knowledge-base entries retrieved by an AI system are relevant to a user’s question. You will review queries and retrieved sources, determine usefulness, and flag irrelevant, outdated, or missing information to improve results.

Responsibilities include assessing content relevance, matching passages to reference documents, and documenting ambiguous or weak sources while providing structured feedback