Web Crawler Engineer — Build Google-scale Crawling + Equity

Exa

Singapore

On-site

SGD 90,000 - 300,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Premium healthcare
Fertility benefits
Monthly wellness stipend

Job summary

Exa in Singapore is hiring a Web Crawler engineer to help build Google-scale crawling for AI agents. You’ll work on distributed crawlers, high-performance languages like C++ or Rust, and modern tooling to crawl the web at massive scale.

You’ll design systems for dynamic content, implement polite crawling, and optimize scheduling for maximum coverage. This is an in-person role in Singapore with international sponsorship and benefits.

Qualifications

  • You have extensive experience building and scaling web crawlers or can ramp up quickly.
  • You have experience with high-performance languages like C++ or Rust.
  • You are familiar with TypeScript and modern web tooling such as Playwright and CDP.

Responsibilities

  • Build a distributed crawler capable of 100M+ pages per day.
  • Optimize crawl politeness and rate limiting across thousands of domains.
  • Design systems to detect dynamic content, JS rendering, and anti-bot measures.
  • Create intelligent crawl scheduling for maximum coverage efficiency.

Skills

Web crawling
C++
Rust
TypeScript
Playwright
CDP (Chrome DevTools Protocol)
Performance optimization

Tools

Playwright
CDP

Job description

Exa in Singapore is hiring a Web Crawler engineer to help build Google-scale crawling for AI agents. You’ll work on distributed crawlers, high-performance languages like C++ or Rust, and modern tooling to crawl the web at massive scale.

You’ll design systems for dynamic content, implement polite crawling, and optimize scheduling for maximum coverage. This is an in-person role in Singapore with international sponsorship and benefits.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Software Engineer, Web Crawling
Software Engineer, Web Crawling

Exa • Singapore

On-site
SGD 90,000 - 300,000
Premium healthcare
Fertility benefits
Monthly wellness stipend
Software Engineer, Backend
Software Engineer, Backend

Exa • Singapore

On-site
SGD 75,000 - 100,000
Premium healthcare benefits
16 weeks fully paid parental leave
Monthly wellness stipend
Forward Deployed Engineer, Singapore
Forward Deployed Engineer, Singapore

Exa • Singapore

On-site
SGD 150,000 - 210,000
Premium healthcare (medical)
Fertility benefits
Monthly wellness stipend
Staff SWE, Ads Data & Measurement, AI-Driven Scale
Staff SWE, Ads Data & Measurement, AI-Driven Scale

GOOGLE ASIA PACIFIC PTE. LTD. • Singapore

On-site
SGD 150,000 - 210,000
Forward Deployed Engineer Intern, 2027
Forward Deployed Engineer Intern, 2027

Google LLC • Singapore

On-site
SGD 20,000 - 27,000
Software Engineer, Ads Platform — High-Scale Backend & ML
Software Engineer, Ads Platform — High-Scale Backend & ML

Google LLC • Singapore

On-site
SGD 120,000 - 180,000
Software Engineer - AI Search Infra & Low-Latency
Software Engineer - AI Search Infra & Low-Latency

Hyphen Partners • Singapore

On-site
SGD 110,000 - 160,000
Full-Stack Web Dev Intern: AI, Automation & Impact
Full-Stack Web Dev Intern: AI, Automation & Impact

NWU • Singapore

On-site
SGD 40,000 - 60,000
Software Engineer (AI/ML)
Software Engineer (AI/ML)

Hyphen Partners • Singapore

On-site
SGD 110,000 - 160,000
Senior Forward Deployed Engineer, Generative AI, Google Cloud (Mandarin)
Senior Forward Deployed Engineer, Generative AI, Google Cloud (Mandarin)

Google LLC • Singapore

On-site
SGD 180,000 - 280,000