Sales Engineering Expert - Evaluator

Mercor; Toronto; On-site; CAD 80,000 - 110,000; Full time

Job summary

Mercor is partnering with a leading AI research organization to devise and apply rigorous grading criteria for real-world pre-sales and solutions engineering artifacts. You will score AI-generated and human samples with detailed justification, ensuring reproducible judgments through structured feedback and iterative reviews.

The role emphasizes deep pre-sales expertise, strong written communication, and experience with technically complex B2B software, including SaaS, cloud, and cybersecurity

Qualifications

• 4+ years of relevant sales engineering experience.
• Fluency in pre-sales, including discovery and tailored demos.
• Experience with complex B2B SaaS or cloud software.

Responsibilities

• Design precise, task-specific grading criteria for real-world pre-sales and solutions engineering deliverables.
• Score AI-generated and human work samples against criteria with written justifications.
• Apply consistent, evidence-based judgment for reproducible scores.
• Incorporate structured feedback from senior reviewers and iterate quickly.

Skills

Sales engineer experience

Technical discovery

Tailored demos

Proof of concept

Product fit evaluation

Handling objections

RFP responses

B2B software

Written communication

Attention to detail

AI training/evaluation experience

Job description

1. Role Overview

Mercor is partnering with a leading AI research organization to engage experienced sales engineers for a project focused on evaluating how well AI systems perform real-world technical sales work. Rather than producing deliverables yourself, you will define what excellent work looks like: designing task-specific grading criteria and scoring completed work samples with rigorous, well-reasoned written justifications.

2. Key Responsibilities

• Design precise, task‑specific grading criteria for real‑world pre‑sales and solutions‑engineering deliverables (technical discovery plans, tailored product demonstrations, proof‑of‑concept and technical evaluation plans, security questionnaires and RFP responses, integration/API/architecture explanations, and product‑fit and technical‑risk assessments)


• Score AI‑generated and human work samples against those criteria, with detailed written justifications for every score


• Apply consistent, evidence‑based judgment so scores are reproducible and defensible


• Incorporate structured feedback from senior reviewers and iterate quickly on your work

3. Ideal Qualifications

• 4+ years of relevant experience as a Sales Engineer, Solutions Engineer, Solutions Consultant, Pre‑Sales Consultant, or Technical Sales Consultant


• Deep fluency in the pre‑sales craft: leading technical discovery, translating customer requirements into product solutions, delivering tailored demos, handling technical objections, supporting proofs of concept, and evaluating product fit and technical risk


• Hands‑on experience with technically complex B2B software — SaaS, cloud infrastructure, cybersecurity, data platforms, developer tools, or enterprise applications


• Exceptionally strong written communication, with the ability to explain exactly why a piece of technical work would or would not land with both technical and business stakeholders


• Detail‑oriented, consistent, and comfortable having your judgment reviewed and calibrated against peers


• Prior experience with AI training, evaluation, or human‑data projects is a strong plus