Remote Information Architect for AI Benchmarking

Meridial

United States

Remote

USD 14,000 - 41,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Aitrainergigs is seeking independent contractors to develop complex evaluation frameworks for AI benchmark tasks in information science, library science, and data governance. You will craft realistic scenarios and rubrics to generate high-quality training data for model evaluation.

The role emphasizes meticulous fact-checking and adherence to data governance best practices, with a remote engagement type and contractor terms. Compensation is negotiated based on experience and location.

Qualifications

  • Demonstrable professional expertise within the information science, library science, data governance, or knowledge management sectors, with a deep understanding of industry standards, metadata schemas, and search methodologies.
  • Strong writing and prompt generation skills, with the ability to design highly realistic, complex information retrieval task scenarios for AI evaluation.
  • Proficiency in rubric generation, specifically the ability to create objective, non-ambiguous scoring criteria that leave no room for subjective interpretation.
  • A meticulous, detail-oriented approach to fact-checking data architectures, search mechanics, and knowledge bases to generate reliable data for system benchmarking.

Responsibilities

  • Task & Prompt Creation: Generating realistic, high-quality prompts for information science, data management, and knowledge organization.
  • Rubric Development: Writing clear, well-structured evaluation rubrics with specific, non-ambiguous criteria.
  • Benchmark Evaluation Data Generation: Producing clean, reliable training data for model evaluation of information architecture and retrieval tasks.
  • Quality Assurance & Fact-Checking: Ensuring tasks and scoring criteria reflect data governance standards and research methodologies.

Skills

Information science
Library science
Data governance
Knowledge management
Prompt generation
Strong writing

Job description

Aitrainergigs is seeking independent contractors to develop complex evaluation frameworks for AI benchmark tasks in information science, library science, and data governance. You will craft realistic scenarios and rubrics to generate high-quality training data for model evaluation.

The role emphasizes meticulous fact-checking and adherence to data governance best practices, with a remote engagement type and contractor terms. Compensation is negotiated based on experience and location.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Government Evaluation Specialist (Contractor)
Remote Government Evaluation Specialist (Contractor)

Meridial • United States

Remote
USD 14,000 - 41,000
Remote Information Specialist: AI Evaluation & Benchmarking
Remote Information Specialist: AI Evaluation & Benchmarking

agency • United States

Remote
USD 14,000 - 41,000
Information Specialist - Freelance AI Trainer Project
Information Specialist - Freelance AI Trainer Project

Meridial • United States

Remote
USD 13,776 - 41,328
Remote Scientific & Technical Service Specialist for AI
Remote Scientific & Technical Service Specialist for AI

Meridial • United States

Remote
USD 14,000 - 41,000
Remote Bioinformatics AI Benchmark Architect
Remote Bioinformatics AI Benchmark Architect

TELUS Digital AI • United States

Remote
USD 69,000 - 138,000
Information Specialist
Information Specialist

Meridial • United States

Remote
USD 14,000 - 41,000
Information Specialist - Freelance AI Trainer Project
Information Specialist - Freelance AI Trainer Project

agency • United States

Remote
USD 14,000 - 41,000
Remote AI Benchmark Specialist — Scientific & Technical Prompts
Remote AI Benchmark Specialist — Scientific & Technical Prompts

Invisible Technologies Inc. • United States

Remote
USD 14,000 - 41,000
Finance & Insurance Evaluation Specialist - Remote
Finance & Insurance Evaluation Specialist - Remote

Meridial • United States

Remote
USD 14,000 - 41,000
Freelance Remote Retail Trade Specialist in AI Benchmarking
Freelance Remote Retail Trade Specialist in AI Benchmarking

Meridial • United States

Remote
USD 14,000 - 41,000