Frontend Code Evaluation Specialist - AI Trainer

Mercor; San Francisco (CA); On-site; USD 60,000 - 90,000; Full time

Job summary

Mercor seeks a skilled frontend reviewer to evaluate real crawled pages and two candidate replicas at 1920×1080, judging fidelity state by state. You will articulate exact reasons for each design and implementation choice, focusing on HTML/CSS structure, typography, and layout relationships.

This is a hands-on, evidence-based assessment task requiring precise documentation and strong English prose. The role emphasizes reading source code, evaluating layout techniques, and producing thoughtful,

Qualifications

• Strong knowledge of hand-written HTML and CSS, semantic markup, flexbox and grid.
• Ability to reason about legacy table-based layouts and floats.
• Experience reading and analyzing page scripts to understand DOM effects.
• Familiarity with setting up a local static server and unzipping archives.
• Excellent written English with evidence-based justification for every decision.

Responsibilities

• Render a reference page and two candidate replicas at 1920×1080 and judge fidelity state by state.
• Diff visual fidelity in detail: box model, spacing, typography, color, borders, images and assets.
• Read the source of both attempts and grade construction quality, distinguishing genuine correctness from superficial accuracy.
• Test responsiveness and explain how the layout behaves at different viewport widths (e.g., 1920px vs 1400px).
• Provide a precise, evidence-based justification for every preference (no vagueness).
• Flag problematic tasks with a clear rationale and escape hatch if necessary.

Skills

Frontend web development

HTML

CSS

DevTools

JavaScript

Local server setup

Documentation

Job description

About the work

We're building a high-quality dataset of human preference judgments on AI-generated frontend code. Each task hands you a reference web page — crawled from the real internet, delivered as a full zipped site tree plus screenshots of its default view and, on some pages, additional states reached by hovering, clicking, or scrolling. Alongside it come two model attempts, A and B, each a zipped self‑contained site tree. The models only ever saw the screenshots; they never had the source.

You download all three, run them locally, view each at a 1920×1080 viewport, interact with them to reach every required state, then open the source of both attempts and grade them against each other — on visual fidelity per state, and on how the code is actually constructed. Structure and responsiveness are explicitly part of the rubric, not just the render.

This is evaluation work, not authoring. The defining skill is not that you can build a page — it's that you can open someone else's page and tell how it was built and where it cheats.

Please read before applying

• Each unit takes roughly 2-3 hours and is timed. This is not microtask work; if you can only offer scattered 15-minute windows, you will not be able to finish a unit.
• You need a real local development environment. A tablet, a Chromebook, or a locked-down work machine that cannot run a local static server will not work for this project.
• You need at least one completed Mercor engagement, delivered in full. We are not onboarding net-new experts to this project.

What you'll do

• Render a reference page and two candidate replications at 1920×1080 and judge which is the closer reproduction, state by state.
• Diff visual fidelity in detail: box model and spacing, typography (family, size, weight, line-height, letter-spacing), color and border treatment, image and asset handling, z‑order and overflow.
• Read the source of both attempts and grade construction quality — distinguishing a replication that is genuinely correct from one that merely looks correct at one viewport. Hardcoded pixel offsets, absolute positioning standing in for real layout, inline style soup, a single undifferentiated div tree, or a screenshot pasted in as an instead of a rebuilt section.
• Test responsiveness: a nav bar that looks right at 1920px but collapses at 1400px is a defect, and you should be able to say precisely why.
• Write a specific, evidence‑cited justification for every preference. We need "B nests the article body in a single absolutely‑positioned div, so the text overlaps the footer below 1600px, while A uses normal document flow" — not "A looks closer."
• Use the "this task is broken" escape hatch with judgment: distinguish a task that genuinely cannot be completed from the screenshots provided from one that is merely hard. Over‑flagging and under‑flagging are both failure modes.

You're a fit if you have

• 3-8 years of professional web development experience, shipping web interfaces for a living, primarily in frontend or full‑stack work.
• Fluency across web eras. The reference pages are real crawled sites — one is a small charter‑fishing business built in the table‑and‑image‑map tradition, another a corporate press‑release page with stacked navigation rows and social share widgets. If your entire career happened inside a modern component framework and you have never authored raw CSS or seen a table used for layout, you will misjudge many of these pages.
• Command of hand‑written HTML and CSS: semantic markup, flexbox, grid, media queries, and legacy float‑ and table‑based layouts you can read and reason about. You should be able to look at a rendered layout and predict what's holding it together before opening DevTools.
• Browser DevTools as muscle memory — setting an exact viewport, walking the element tree, checking computed styles, watching what a hover handler mutates.
• Command‑line comfort: unzipping an archive, standing up a static local server because the relative asset paths demand it, and untangling a broken image reference rather than giving up and grading from the screenshot.
• Enough JavaScript to read a page's scripts and understand what they do to the DOM, even if you don't write JS daily.
• Professional written English. Every task ends in a free‑text justification, and a rating without a specific rationale is worth very little.

Equipment

• A desktop or laptop that displays a 1920×1080 viewport.
• Administrator rights on your own machine, so you can install and run a local server.

Nice to have

• Prior RLHF, preference‑labeling, model‑evaluation, or structured code‑review work — the strongest single signal. Rubric‑driven comparison at volume needs almost no ramp here.
• Pixel‑perfect design‑to‑code experience: agency work, design systems, template production. Anyone who has had a designer reject a build over four pixels has exactly the fidelity eye this needs.
• Accessibility expertise (ARIA, semantic landmarks, heading hierarchy) — you'll notice immediately when an attempt renders a heading as a styled .
• Familiarity with how LLMs fail at code generation.
• Web scraping, archiving, or DOM‑parsing background — comfort with messy crawled site trees.
• More than one completed Mercor project, and availability in contiguous multi‑hour blocks.

Note:

This seat is for practicing web developers. Backend‑only, ML/data‑science‑only, mobile‑native‑only, and DevOps‑only engineers do not have the UI instincts this requires, however strong they are otherwise. Designers who do not code cannot grade the source axes at all. Framework‑only engineers who have never authored CSS outside a component library will struggle with the legacy reference pages.