Remote Web Access Engineer (Crawler) - Scalable Fetching

Applied Methods Ltd

Greater London

Remote

GBP 70,000 - 100,000

Full time

6 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Perplexity is hiring a Web Access Engineer for the Crawler team to build reliable fetching and rendering systems for AI-powered search. You will diagnose network errors, page-loading issues, and incomplete rendering, working across DNS, TLS, HTTP, caching, and client-side execution.

You will own end-to-end improvements, create regression tests, and define metrics for fetching reliability and content completeness at scale in London with remote options.

Qualifications

  • 5+ years of software engineering in web infrastructure or crawling.
  • Experience with HTTP/1.1, HTTP/2, HTTP/3 and TLS.
  • Proficiency in Go, Python, Rust or C++.
  • Familiarity with CDNs, caching and rate limits.

Responsibilities

  • Own fetching and rendering improvements end to end from investigation to production release.
  • Investigate fetch and rendering failures across DNS, TLS, HTTP, caching, redirects and client-side rendering.
  • Improve page loading, completion detection and content capture for dynamic pages.
  • Define metrics for fetching reliability and content completeness; build regression tests.

Skills

Go
Python
Rust
C++
Chrome DevTools
HAR files
CDP
curl

Tools

Chrome DevTools
HAR files
CDP
curl

Job description

Perplexity is hiring a Web Access Engineer for the Crawler team to build reliable fetching and rendering systems for AI-powered search. You will diagnose network errors, page-loading issues, and incomplete rendering, working across DNS, TLS, HTTP, caching, and client-side execution.

You will own end-to-end improvements, create regression tests, and define metrics for fetching reliability and content completeness at scale in London with remote options.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Web Access Engineer - Crawler Infrastructure
Senior Web Access Engineer - Crawler Infrastructure

Perplexity AI • Greater London

On-site
GBP 85,000 - 140,000
Member of Technical Staff (Software Engineer, Fraud Prevention)
Member of Technical Staff (Software Engineer, Fraud Prevention)

Perplexity AI • Greater London

On-site
GBP 85,000 - 140,000
1d Perplexity Member of Technical Staff (Web Access Engineer, Crawler) London Perplexity 1d Member of Technical Staff (Web Access Engineer, Crawler) London
1d Perplexity Member of Technical Staff (Web Access Engineer, Crawler) London Perplexity 1d Member of Technical Staff (Web Access Engineer, Crawler) London

Applied Methods Ltd • Greater London

Remote
GBP 70,000 - 100,000
Backend Engineer - Web Crawling & Search Platform
Backend Engineer - Web Crawling & Search Platform

Perplexity • Greater London

On-site
GBP 90,000 - 120,000
Remote: Research Engineer, Large-Scale Web Crawling
Remote: Research Engineer, Large-Scale Web Crawling

ElevenLabs • Greater London

Hybrid
GBP 80,000 - 120,000
Innovative culture
Growth paths
Learning & development
+3
Member of Technical Staff (Site Reliability Engineer, Crawler)
Member of Technical Staff (Site Reliability Engineer, Crawler)

Academic Key • Greater London

On-site
GBP 90,000 - 150,000
Remote Research Engineer - Large-Scale Web Crawling & Data Pipelines
Remote Research Engineer - Large-Scale Web Crawling & Data Pipelines

Devconnectplatform • Greater London

Hybrid
GBP 90,000 - 143,000
Annual discretionary stipend
Annual company offsite
Co-working stipend
+1
1d Perplexity Member of Technical Staff (Site Reliability Engineer, Crawler) London Perplexity 1d Member of Technical Staff (Site Reliability Engineer, Crawler) London
1d Perplexity Member of Technical Staff (Site Reliability Engineer, Crawler) London Perplexity 1d Member of Technical Staff (Site Reliability Engineer, Crawler) London

Applied Methods Ltd • Greater London

On-site
GBP 110,000 - 140,000
Staff Software Engineer, Web-Scale Crawling & Ingestion
Staff Software Engineer, Web-Scale Crawling & Ingestion

FDE Times • Greater London

Hybrid
GBP 120,000 - 190,000
Competitive compensation
Career growth and learning
Flexibility and ownership
+3
Senior SRE: Crawler & Search Infrastructure
Senior SRE: Crawler & Search Infrastructure

Applied Methods Ltd • Greater London

On-site
GBP 110,000 - 140,000