Member of Technical Staff (Site Reliability Engineer, Crawler)

Perplexity

London

On-site

CAD 120,000 - 160,000

Full time

5 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Maven Ventures’ crawler team is seeking an experienced SRE to improve throughput and reliability at scale. You will optimize Kubernetes workloads, define capacity plans, and help deploy safer rollouts that minimize risk while improving observability.

You will own incident response, build tooling, and drive metrics across latency, availability, and resource usage. This role emphasizes pragmatic reliability and durable fixes with cross-team collaboration.

Qualifications

  • 5+ years in SRE, production engineering, or infrastructure-heavy backend roles.
  • Strong programming skills in Python, C++, Rust, or Go.
  • Cloud infrastructure, production Kubernetes, and IaC like Terraform.
  • Solid Linux and networking fundamentals (TCP/IP, DNS, TLS, HTTP, load balancing).
  • Experience building observability with Datadog, Prometheus, or Grafana and using SLOs.

Responsibilities

  • Operate and optimize Kubernetes workloads and supporting infrastructure for crawler services.
  • Build and maintain infrastructure as code and CI/CD pipelines with safe rollouts and quick rollback.
  • Define SLOs and build monitoring, alerting, and observability for availability, latency, throughput, and saturation.
  • Own capacity planning and diagnose queue backlogs, network failures, and bottlenecks with service owners.
  • Participate in on-call rotation, lead incident response and postmortems to durable improvements.
  • Write production software to automate diagnosis and routine remediation with validation gates.

Skills

Python
C++
Rust
Go
Kubernetes
Terraform
Linux
Networking
Datadog
Prometheus

Tools

Kubernetes
Terraform

Job description

Keeping a crawler running is not the same as keeping it productive. Queues can grow while servers look healthy, and a small change can become a fleet-wide bottleneck. On Perplexity's Crawler team, you'll improve throughput, plan capacity, and build operational reliability through safer deployments, better observability, and durable fixes.

Responsibilities
  • Operate and optimize Kubernetes workloads and supporting infrastructure for crawler services, including fetching and rendering fleets.
  • Build and maintain infrastructure as code and CI/CD pipelines, with safe rollouts, validation, and fast rollback.
  • Define SLOs and build monitoring, alerting, and observability for availability, latency, throughput, and resource saturation.
  • Own capacity planning and resource efficiency; diagnose queue backlogs, network failures, and resource bottlenecks with service owners.
  • Participate in the team's on-call rotation, lead incident response, and turn blameless postmortems into durable reliability improvements.
  • Write production software to automate diagnosis and routine remediation, with validation and approval gates for high-impact actions.
Qualifications
  • 5+ years of experience in SRE, production engineering, or infrastructure-heavy backend roles, with strong programming skills in Python, C++, Rust, or Go.
  • Hands-on experience with cloud infrastructure, production Kubernetes, infrastructure as code such as Terraform, and automated deployment workflows.
  • Strong Linux and networking fundamentals, including TCP/IP, DNS, TLS, HTTP, and load balancing.
  • Experience building observability with tools such as Datadog, Prometheus, or Grafana, and using SLOs to guide reliability improvements.
  • Proven experience diagnosing distributed-system failures, leading production incidents, and improving system performance or resource efficiency.
Nice to have
  • Experience operating web crawlers, search infrastructure, or browser fleets at scale.
  • Experience with distributed queues, backpressure, workload isolation, or multi-region recovery.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Member of Technical Staff (Web Access Engineer, Crawler)
Member of Technical Staff (Web Access Engineer, Crawler)

Perplexity • London

On-site
CAD 110,000 - 160,000
Web Crawler & Fetch/Render Reliability Engineer
Web Crawler & Fetch/Render Reliability Engineer

Perplexity • London

On-site
CAD 110,000 - 160,000
Site Reliability Engineer (AI Platform)
Site Reliability Engineer (AI Platform)

OpenTable • Toronto

On-site
CAD 140,000 - 190,000
Generous parental leave
Generous paid vacation
Paid volunteer time
+3
Site Reliability Engineer
Site Reliability Engineer

Chair.com.pk • Toronto

On-site
CAD 120,000 - 180,000
Sr Engineer, Infrastructure and Tooling
Sr Engineer, Infrastructure and Tooling

hireVouch • Toronto

On-site
CAD 120,000 - 180,000
30 days PTO
Health, dental & wellness benefits
Tech allowance
+1
Service Reliability Engineer
Service Reliability Engineer

Deliverect • Toronto

On-site
CAD 90,000 - 130,000
Devops linux
Devops linux

HireOn Tech • Montreal

On-site
CAD 90,000 - 140,000
Senior Site Reliability Engineer
Senior Site Reliability Engineer

Compunnel Inc. • Montreal

On-site
CAD 100,000 - 130,000
Senior Site Reliability Engineer
Senior Site Reliability Engineer

Orion Innovation • British Columbia

On-site
CAD 100,000 - 130,000
Senior Site Reliability Specialist (SRE)
Senior Site Reliability Specialist (SRE)

AlayaCare • Montreal

On-site
CAD 110,000 - 150,000