Architecture AI Model Evaluator & Task Designer

mercor; United States; Remote; USD 62,000 - 83,000; Full time

Job summary

Mercor is seeking an Architecture Expert to evaluate AI models in visual document understanding within the Architecture domain. The role involves authoring grounded tasks with clear rubrics and collaborating with SMEs to improve dataset quality.

The contractor role is remote, with a commitment of 15–20 hours per week, at a rate of $50/hour. Flexible scheduling and asynchronous work are expected to meet deadlines while advancing model performance.

Qualifications

• Expertise in the Architecture domain.
• Experience in authoring complex tasks with objective rubrics.

Responsibilities

• Evaluate AI models on visual document understanding in the Architecture domain.
• Author complex, grounded tasks with clear ground-truth outputs and objective rubrics.
• Review and improve the accuracy of AI-generated content to align with domain principles.
• Collaborate with subject matter experts to enhance dataset consistency and coverage.
• Work independently and asynchronously to meet deadlines and improve AI model performance.

Skills

Architecture domain expertise

Task authoring with rubrics

Job description

Mercor is seeking an Architecture Expert to evaluate AI models in visual document understanding within the Architecture domain. The role involves authoring grounded tasks with clear rubrics and collaborating with SMEs to improve dataset quality.

The contractor role is remote, with a commitment of 15–20 hours per week, at a rate of $50/hour. Flexible scheduling and asynchronous work are expected to meet deadlines while advancing model performance.