Remote Research Engineer: Large-Scale Web Crawling

Valor Capital Group; New York (NY); Remote; USD 140,000 - 210,000; Full time

Job summary

ElevenLabs is seeking a Research Engineer to join the research team, focusing on large-scale web crawling for frontier AI models. You will own crawling systems that source world-class data from the open web, and you will thrive on solving hard crawling problems at scale.

You will design targeted pipelines to locate high-value data sources and build tooling for researchers to access freshly crawled data efficiently and reliably, enabling rapid experimentation across language, audio, and video

Qualifications

• Hands-on experience building and scaling web crawlers or scraping systems, ideally for ML training data.
• Strong engineering skills in distributed systems at scale (Kubernetes, queue-based architectures, or pipelines).
• The capacity to autonomously evaluate the quality, coverage, and compliance of crawled data and to build tooling to measure it.

Responsibilities

• Build and operate large-scale distributed web crawlers that fetch and extract data from billions of pages.
• Address challenges like messy HTML data extraction, deduplication, freshness, and politeness/rate limiting.
• Design crawling pipelines to find high-value data sources including audio, video, and multilingual content.
• Create tooling to request, monitor, and explore newly crawled web data quickly and reliably.

Skills

Web crawlers

Distributed systems

Data quality evaluation

Tools

Kubernetes

Queue-based architectures

Job description

ElevenLabs is seeking a Research Engineer to join the research team, focusing on large-scale web crawling for frontier AI models. You will own crawling systems that source world-class data from the open web, and you will thrive on solving hard crawling problems at scale.

You will design targeted pipelines to locate high-value data sources and build tooling for researchers to access freshly crawled data efficiently and reliably, enabling rapid experimentation across language, audio, and video