Materials Science Task Architect for AI Evaluation

Mercor; San Francisco (CA); On-site; USD 120,000 - 170,000; Full time

Job summary

Mercor is seeking a senior scientist/engineer to design tasks in materials science and engineering that test AI models performing expert work. You write realistic prompts, assemble a data room, and define grading criteria to make evaluation objective.

You will run tasks against prompts, analyze results, and iterate until the model’s reasoning can be clearly distinguished from incorrect approaches, collaborating with the project lead and domain experts.

Qualifications

• Senior expertise in metals, ceramics, composites, characterization, additive manufacturing or in electrical engineering (analog, RF, power) or in mechanical engineering (design, CFD and fluids, heat transfer, pressure vessels, NDE, CAE).
• Comfortable navigating ambiguity — much of the work is still being defined, and shaping it is part of the role.
• Able to explain not just the right answer, but why a wrong answer is wrong.

Responsibilities

• Author domain-authentic tasks drawn from work you've actually done.
• Build the data room and grading criteria that make the task objectively scorable.
• Run the task against the model, analyse where it succeeds or fails, and iterate until it genuinely discriminates.
• Work with the project lead and other domain experts to settle what "good" looks like in your domain.

Skills

Materials science

Electrical engineering

Mechanical engineering

Ambiguity navigation

Explain reasoning

Job description

Mercor is seeking a senior scientist/engineer to design tasks in materials science and engineering that test AI models performing expert work. You write realistic prompts, assemble a data room, and define grading criteria to make evaluation objective.

You will run tasks against prompts, analyze results, and iterate until the model’s reasoning can be clearly distinguished from incorrect approaches, collaborating with the project lead and domain experts.