Evaluation Systems Engineer — ML Metrics & Pipelines

Mendable; San Francisco (CA); Hybrid; USD 210,000 - 275,000; Full time

Benefits offered by this job

Competitive equity

PTO 15 days

Parental leave 12 weeks

Wellness stipend

Learning & Development

Team offsites

Sabbatical

401(k) plan

Pet insurance

Job summary

Firecrawl is the easiest way to turn the web into data AI agents can use. One API call converts any URL into clean, LLM-ready markdown or structured data - the boring-hard problem everyone building with LLMs eventually hits, solved.

You’ll design the metrics, build the pipelines, generate the datasets, and own the feedback loop from output quality back to model and product decisions. If you care about what “good” means and can measure it at scale, this is the role.

Qualifications

• Experience: 4+ years in ML, research engineering, or data-heavy backend, with real evaluation work

Responsibilities

• Design the metrics that define what 'good output' actually means across millions of sites, formats, and edge cases
• Build the pipelines and harnesses that measure quality rigorously and at scale
• Generate and curate the datasets that make evaluation trustworthy
• Own the feedback loop from output quality back to model and product decisions
• Turn 'did that work?' into an answer the whole team can act on

Skills

4+ years experience

ML

research engineering

Job description

Firecrawl is the easiest way to turn the web into data AI agents can use. One API call converts any URL into clean, LLM-ready markdown or structured data - the boring-hard problem everyone building with LLMs eventually hits, solved.

You’ll design the metrics, build the pipelines, generate the datasets, and own the feedback loop from output quality back to model and product decisions. If you care about what “good” means and can measure it at scale, this is the role.