Architecture AI Evaluation & Task Designer

mercor; San Francisco (CA); Hybrid; CAD 58,000 - 80,000; Part time

Job summary

Mercor is seeking an Architecture Expert for a remote, part-time contract role. The position involves evaluating AI models focused on visual document understanding within the Architecture domain and authoring tasks with clear rubrics.

Collaborate with domain experts to ensure dataset quality and model alignment. You’ll work independently, 15–20 hours per week, contributing to improving AI performance while meeting project deadlines.

Qualifications

• Expertise in Architecture domain.
• Experience in authoring complex tasks with objective rubrics.

Responsibilities

• Evaluate AI models on visual document understanding in the Architecture domain.
• Author complex, grounded tasks with clear ground-truth outputs and objective rubrics.
• Review and improve the accuracy of AI-generated content to ensure alignment with domain principles.
• Collaborate with subject matter experts to enhance dataset consistency and coverage.
• Work independently and asynchronously to meet deadlines and improve AI model performance.

Skills

Architecture domain expertise

Task authoring with rubrics

Job description

Mercor is seeking an Architecture Expert for a remote, part-time contract role. The position involves evaluating AI models focused on visual document understanding within the Architecture domain and authoring tasks with clear rubrics.

Collaborate with domain experts to ensure dataset quality and model alignment. You’ll work independently, 15–20 hours per week, contributing to improving AI performance while meeting project deadlines.