Remote Data Infrastructure Engineer - Web Crawling Pipelines

Unknown; United States; Remote; USD 160,000 - 250,000; Full time

Benefits offered by this job

Fully remote role

Job summary

Unknown builds large-scale web data systems for AI research and model development. The team focuses on reliable data collection and dataset curation to power cutting-edge research.

We are seeking engineers to build and maintain web crawlers across diverse domains, designing high-throughput, fault-tolerant pipelines. Remote work is fully supported for this role, with a compensation range of $160,000–$250,000 per year.

Qualifications

• 2+ years of experience in web crawling or large-scale pipelines.
• Proficient in Go, Rust, Python, Java, or C++.

Responsibilities

• Build and maintain web crawlers across diverse domains.
• Design high-throughput, fault-tolerant collection systems.

Skills

Go

Rust

Python

Java

C++

Job description

Unknown builds large-scale web data systems for AI research and model development. The team focuses on reliable data collection and dataset curation to power cutting-edge research.

We are seeking engineers to build and maintain web crawlers across diverse domains, designing high-throughput, fault-tolerant pipelines. Remote work is fully supported for this role, with a compensation range of $160,000–$250,000 per year.