Senior Materials Science Task Architect

Mercor; San Francisco (CA); Remote; USD 140,000 - 190,000; Full time

Job summary

Mercor in San Francisco is seeking a senior evaluator to design tasks that test AI models performing real expert work. You will craft prompts, assemble a data room, and establish scoring rubrics that reflect authentic domain practice.

You will collaborate with a project lead and domain experts to define what good looks like, iterating until the model's reasoning is robustly challenged across materials science, electrical, or mechanical engineering contexts.

Qualifications

• Senior expertise in materials science, electrical engineering, or mechanical engineering.
• Ability to navigate ambiguity and shape early-stage projects.
• Strong ability to explain reasoning for answers and errors.

Responsibilities

• Author realistic tasks including prompts, data rooms, and grading criteria.
• Build data rooms and objective rubrics to score model performance.
• Run tasks against models, analyze results, and iterate to sharpen discrimination.
• Collaborate with project lead and domain experts to define success criteria.

Skills

Materials science

Electrical engineering

Mechanical engineering

Job description

Mercor in San Francisco is seeking a senior evaluator to design tasks that test AI models performing real expert work. You will craft prompts, assemble a data room, and establish scoring rubrics that reflect authentic domain practice.

You will collaborate with a project lead and domain experts to define what good looks like, iterating until the model's reasoning is robustly challenged across materials science, electrical, or mechanical engineering contexts.