AI-Driven Solutions Engineer Evaluator

Obsidian; San Francisco (CA); Remote; USD 110,000 - 160,000; Full time

Job summary

Mercor is seeking experienced sales engineers to define grading criteria for pre‑sales deliverables and to score AI‑generated and human work samples with detailed written justifications. You will establish evidence‑based scoring criteria and calibrate judgments across technical discovery plans, demos, proofs of concept, and evaluation plans.

The role emphasizes clear written communication, consistency, and the ability to iterate quickly based on feedback from senior reviewers.

Qualifications

• 4+ years of relevant experience as a Sales Engineer, Solutions Engineer, Solutions Consultant, Pre-Sales Consultant, or Technical Sales Consultant.

Responsibilities

• Design precise, task‑specific grading criteria for real‑world pre‑sales and solutions‑engineering deliverables (technical discovery plans, tailored product demonstrations, proof‑of‑concept and technical evaluation plans, security questionnaires and RFP responses, integration/API/architecture explanations, and product‑fit and technical‑risk assessments) .
• Score AI‑generated and human work samples against those criteria, with detailed written justifications for every score
• Apply consistent, evidence‑based judgment so scores are reproducible and defensible
• Incorporate structured feedback from senior reviewers and iterate quickly on your work

Skills

Sales engineering

Technical discovery

Solutions demos

Written communication

Attention to detail

Judgment calibration

Job description

Mercor is seeking experienced sales engineers to define grading criteria for pre‑sales deliverables and to score AI‑generated and human work samples with detailed written justifications. You will establish evidence‑based scoring criteria and calibrate judgments across technical discovery plans, demos, proofs of concept, and evaluation plans.

The role emphasizes clear written communication, consistency, and the ability to iterate quickly based on feedback from senior reviewers.