Materials Science Expert - Build AI Evaluation Tasks

Obsidian; San Francisco (CA); On-site; USD 150,000 - 190,000; Full time

Job summary

Obsidian is seeking a senior engineer/researcher to design realistic evaluation tasks in materials science or engineering. You will author prompts, assemble data rooms, and create objective grading criteria to test AI models' expert performance.

You’ll collaborate with a project lead and domain experts, navigating ambiguity and explaining not just the right answer but why alternatives are incorrect, aligning tests with real-world specifications.

Qualifications

• Senior expertise in materials science — metals, ceramics, composites, characterization, additive manufacturing — or in electrical engineering (analog, RF, power) or mechanical engineering (design, CFD and fluids, heat transfer, pressure vessels, NDE, CAE).
• Comfortable navigating ambiguity and shaping undefined work.
• Able to explain not just the right answer, but why a wrong answer is wrong.

Responsibilities

• Author domain-authentic tasks drawn from work you've actually done.
• Build the data room and grading criteria that make the task objectively scorable.
• Run the task against the model, analyse where it succeeds or fails, and iterate until it genuinely discriminates.
• Work with the project lead and other domain experts to settle what 'good' looks like in your domain.

Skills

Materials science

Electrical engineering

Mechanical engineering

Job description

Obsidian is seeking a senior engineer/researcher to design realistic evaluation tasks in materials science or engineering. You will author prompts, assemble data rooms, and create objective grading criteria to test AI models' expert performance.

You’ll collaborate with a project lead and domain experts, navigating ambiguity and explaining not just the right answer but why alternatives are incorrect, aligning tests with real-world specifications.