Senior Data Infrastructure Engineer - Indexing & Pipelines

Perplexity

Greater London

Presencial

GBP 90 000 - 140 000

Tempo integral

Há 5 dias
Torna-te num dos primeiros candidatos

Uma candidatura completa num minuto

Currículo e carta de apresentação personalizados, prontos a enviar.

Ultrapassa os filtros ATS

Resumo da oferta

Perplexity is seeking an engineer to work on the indexing and data processing layer of its Search Crawler. You will design, implement, and operate large-scale indexing systems and data pipelines, focusing on offline and nearline workflows to keep data fresh, complete, and queryable.

You will collaborate with retrieval, ranking, and ML teams to meet downstream requirements, optimize storage formats, and ensure observability and reliability under high throughput.

Qualificações

  • 5+ years of experience building production backend or data infrastructure systems.
  • Strong programming experience in Go, C++, or Rust.
  • Experience with high-throughput data processing systems and very large datasets.
  • Experience building or operating databases, storage systems, data planes, or indexing pipelines.
  • Strong understanding of distributed systems, fault tolerance, consistency, and scalability.
  • Experience running production systems and handling operational incidents.
  • A systems-thinking mindset and the ability to reason about end-to-end data flows.

Responsabilidades

  • Design, implement, and operate large-scale indexing systems and data pipelines at the core of Perplexity's search infrastructure.
  • Develop and optimize indexing strategies that balance performance, freshness, and resource efficiency.
  • Work on storage formats, compaction, and update mechanisms that keep data accessible and current.
  • Keep pipelines reliable and predictable under high-throughput conditions.
  • Build well-tested components with clear responsibilities and interface contracts that stay flexible as the system evolves.
  • Define and implement observability across offline and near line pipelines, including structured logs, metrics, and data quality signals.
  • Monitor throughput, resource usage, and cost, and drive optimisations as business needs change.
  • Partner with retrieval, ranking, and ML teams to make sure indexing outputs meet their requirements.
  • Enable safe experimentation on indexing strategies and processing logic through controlled rollouts and clearly defined quality signals.

Conhecimentos

Systems thinking
Distributed systems
Operational incidents handling
End-to-end data flows

Ferramentas

Go
C++
Rust

Descrição da oferta de emprego

Perplexity is seeking an engineer to work on the indexing and data processing layer of its Search Crawler. You will design, implement, and operate large-scale indexing systems and data pipelines, focusing on offline and nearline workflows to keep data fresh, complete, and queryable.

You will collaborate with retrieval, ranking, and ML teams to meet downstream requirements, optimize storage formats, and ensure observability and reliability under high throughput.

Obtém a tua avaliação gratuita e confidencial do currículo.

ou arrasta e larga o ficheiro aqui.

Similar jobs

Ofertas semelhantes que vale a pena comparar

Staff Engineer: Large-Scale Data Indexing & Pipelines
Staff Engineer: Large-Scale Data Indexing & Pipelines

Perplexity • Greater London

Presencial
GBP 90 000 - 140 000
Member of Technical Staff (Index Engineer, Search Crawler)
Member of Technical Staff (Index Engineer, Search Crawler)

Perplexity • Greater London

Presencial
GBP 90 000 - 140 000
Member of Technical Staff (Index, Search Crawler)
Member of Technical Staff (Index, Search Crawler)

Perplexity • Greater London

Presencial
GBP 90 000 - 140 000
Member of Technical Staff (Index Engineer, Search Crawler)
Member of Technical Staff (Index Engineer, Search Crawler)

ResumeXrays LLC • Greater London

Presencial
GBP 90 000 - 130 000
Senior Data Infra Engineer – Scalable Indexing
Senior Data Infra Engineer – Scalable Indexing

ResumeXrays LLC • Greater London

Presencial
GBP 90 000 - 130 000
Senior Systems Engineer, Search Core & Indexing
Senior Systems Engineer, Search Core & Indexing

Perplexity • Greater London

Presencial
GBP 90 000 - 130 000
Backend Engineer - Web Crawling & Search Platform
Backend Engineer - Web Crawling & Search Platform

Perplexity • Greater London

Presencial
GBP 90 000 - 120 000
Senior Search Core Engineer: High-Scale, Low-Level Systems
Senior Search Core Engineer: High-Scale, Low-Level Systems

Perplexity • Greater London

Presencial
GBP 90 000 - 130 000
Member of Technical Staff (Search Engine Developer, Search Core)
Member of Technical Staff (Search Engine Developer, Search Core)

Perplexity • Greater London

Presencial
GBP 90 000 - 130 000
Senior Site Reliability Engineer, Search Crawler Infra
Senior Site Reliability Engineer, Search Crawler Infra

Perplexity • Greater London

Presencial
GBP 90 000 - 120 000