Crawler Web Access Engineer: Reliable Fetch & Render

Perplexity

Greater London

On-site

GBP 80,000 - 110,000

Full time

9 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Perplexity is hiring a Web Access Engineer for the Crawler team to build reliable fetching and rendering systems for AI-powered search. You will diagnose network errors, page-loading issues, and incomplete rendering, tracing problems from network protocols through client-side execution to production fixes that improve content quality at scale.

You will own end-to-end fetching and rendering improvements, investigate failures across DNS, TLS, HTTP, and caching, and develop regression tests.

Qualifications

  • 5+ years of software engineering experience in web infrastructure, browsers, CDNs, or crawling.
  • Deep understanding of HTTP/1.1, HTTP/2, and HTTP/3 with caching, redirects, and TLS knowledge.
  • Experience debugging JavaScript and using Chrome DevTools, CDP, HAR, curl, and packet captures.
  • Comfort working independently when the path forward is not fully defined.

Responsibilities

  • Own fetching and rendering improvements end to end from investigation to production rollout.
  • Investigate fetch and rendering failures across DNS, TLS, HTTP, caching, redirects, and client-side rendering.
  • Improve page loading, completion detection, and content capture to preserve relevant content.
  • Ensure crawler respects site-imposed access restrictions and rate limits, and retries only transient errors.
  • Build tools to reproduce failures, compare crawler output against expected content, and create regression tests.
  • Define metrics for fetching reliability and content completeness to guide improvements.

Skills

Go
Python
Rust
C++
HTTP/1.1/2/3
TLS/DNS
JavaScript debugging

Tools

Chrome DevTools
CDP
HAR files
curl
packet captures

Job description

Perplexity is hiring a Web Access Engineer for the Crawler team to build reliable fetching and rendering systems for AI-powered search. You will diagnose network errors, page-loading issues, and incomplete rendering, tracing problems from network protocols through client-side execution to production fixes that improve content quality at scale.

You will own end-to-end fetching and rendering improvements, investigate failures across DNS, TLS, HTTP, and caching, and develop regression tests.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Web Access Engineer (Crawler) - Scalable Fetching
Remote Web Access Engineer (Crawler) - Scalable Fetching

Applied Methods Ltd • Greater London

Remote
GBP 70,000 - 100,000
Senior Web Access Engineer - Crawler Infrastructure
Senior Web Access Engineer - Crawler Infrastructure

Perplexity AI • Greater London

On-site
GBP 85,000 - 140,000
Member of Technical Staff (Web Access Engineer, Search Crawler)
Member of Technical Staff (Web Access Engineer, Search Crawler)

Perplexity • Greater London

On-site
GBP 80,000 - 110,000
Member of Technical Staff (Software Engineer, Fraud Prevention)
Member of Technical Staff (Software Engineer, Fraud Prevention)

Perplexity AI • Greater London

On-site
GBP 85,000 - 140,000
1d Perplexity Member of Technical Staff (Web Access Engineer, Crawler) London Perplexity 1d Member of Technical Staff (Web Access Engineer, Crawler) London
1d Perplexity Member of Technical Staff (Web Access Engineer, Crawler) London Perplexity 1d Member of Technical Staff (Web Access Engineer, Crawler) London

Applied Methods Ltd • Greater London

Remote
GBP 70,000 - 100,000
Backend Engineer - Web Crawling & Search Platform
Backend Engineer - Web Crawling & Search Platform

Perplexity • Greater London

On-site
GBP 90,000 - 120,000
Member of Technical Staff (Site Reliability Engineer, Crawler)
Member of Technical Staff (Site Reliability Engineer, Crawler)

Perplexity • Greater London

On-site
GBP 90,000 - 125,000
Remote: Research Engineer, Large-Scale Web Crawling
Remote: Research Engineer, Large-Scale Web Crawling

ElevenLabs • Greater London

Hybrid
GBP 80,000 - 120,000
Innovative culture
Growth paths
Learning & development
+3
Senior Site Reliability Engineer, Search Crawler Infra
Senior Site Reliability Engineer, Search Crawler Infra

Perplexity • Greater London

On-site
GBP 90,000 - 120,000
Member of Technical Staff (Site Reliability Engineer, Crawler)
Member of Technical Staff (Site Reliability Engineer, Crawler)

Academic Key • Greater London

On-site
GBP 90,000 - 150,000