AI Task Designer — Materials & Mechanical Engineering

Obsidian; San Francisco (CA); Remote; USD 140,000 - 210,000; Full time

Job summary

Obsidian in San Francisco is seeking a senior professional to design realistic AI evaluation tasks, including prompts, a data room, and objective grading criteria. You will author authentic tasks drawn from actual work and build rigorous benchmarks to distinguish model performance.

You'll collaborate with project leads and domain experts to define what constitutes good results and iterate until the criteria reliably discriminate model behavior.

Qualifications

• Senior expertise in materials science or related engineering disciplines.
• Comfortable navigating ambiguity and shaping evolving work.
• Ability to explain why a correct answer is correct and why others are wrong.

Responsibilities

• Author domain-authentic tasks drawn from work you've actually done.
• Build the data room and grading criteria that make the task objectively scorable.
• Run the task against the model, analyse where it succeeds or fails, and iterate until it genuinely discriminates.
• Work with the project lead and other domain experts to settle what "good" looks like in your domain.

Skills

Materials science

Electrical engineering

Mechanical engineering

Job description

Obsidian in San Francisco is seeking a senior professional to design realistic AI evaluation tasks, including prompts, a data room, and objective grading criteria. You will author authentic tasks drawn from actual work and build rigorous benchmarks to distinguish model performance.

You'll collaborate with project leads and domain experts to define what constitutes good results and iterate until the criteria reliably discriminate model behavior.