Senior Data Infra Engineer – Scalable Indexing

ResumeXrays LLC

Greater London

On-site

GBP 90,000 - 130,000

Full time

4 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

ResumeXrays LLC is seeking an experienced data infrastructure engineer to design and operate large-scale indexing systems and data pipelines. You will work on offline and nearline processing to ingest, transform, and organize vast crawled content into efficient, queryable structures, ensuring freshness and reliability.

You will collaborate with ML and retrieval teams to align indexing outputs with downstream needs, optimize storage formats and update mechanisms, and implement robust

Qualifications

  • 5+ years of experience building production backend or data infrastructure systems.
  • Strong programming experience in Go, C++, or Rust.
  • Experience with high-throughput data processing systems and very large datasets.
  • Experience building or operating databases, storage systems, data planes, or indexing pipelines.
  • Strong understanding of distributed systems, fault tolerance, consistency, and scalability.
  • Experience running production systems and handling operational incidents.
  • A systems-thinking mindset and the ability to reason about end-to-end data flows.

Responsibilities

  • Design, implement, and operate large-scale indexing systems and data pipelines at the core of Perplexity's search infrastructure.
  • Develop and optimize indexing strategies that balance performance, freshness, and resource efficiency.
  • Work on storage formats, compaction, and update mechanisms that keep data accessible and current.
  • Keep pipelines reliable and predictable under high-throughput conditions.
  • Build well-tested components with clear responsibilities and interface contracts that stay flexible as the system evolves.
  • Define and implement observability across offline and near line pipelines, including structured logs, metrics, and data quality signals.
  • Monitor throughput, resource usage, and cost, and drive optimisations as business needs change.
  • Partner with retrieval, ranking, and ML teams to make sure indexing outputs meet their requirements.
  • Enable safe experimentation on indexing strategies and processing logic through controlled rollouts and clearly defined quality signals.

Skills

Go
C++
Rust
Distributed systems
Data pipelines

Tools

Databases
Storage systems
Streaming systems

Job description

ResumeXrays LLC is seeking an experienced data infrastructure engineer to design and operate large-scale indexing systems and data pipelines. You will work on offline and nearline processing to ingest, transform, and organize vast crawled content into efficient, queryable structures, ensuring freshness and reliability.

You will collaborate with ML and retrieval teams to align indexing outputs with downstream needs, optimize storage formats and update mechanisms, and implement robust

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Infrastructure Engineer - Indexing & Pipelines
Senior Data Infrastructure Engineer - Indexing & Pipelines

Perplexity • Greater London

On-site
GBP 90,000 - 140,000
Staff Engineer: Large-Scale Data Indexing & Pipelines
Staff Engineer: Large-Scale Data Indexing & Pipelines

Perplexity • Greater London

On-site
GBP 90,000 - 140,000
Member of Technical Staff (Index Engineer, Search Crawler)
Member of Technical Staff (Index Engineer, Search Crawler)

Perplexity • Greater London

On-site
GBP 90,000 - 140,000
Member of Technical Staff (Index, Search Crawler)
Member of Technical Staff (Index, Search Crawler)

Perplexity • Greater London

On-site
GBP 90,000 - 140,000
Member of Technical Staff (Index Engineer, Search Crawler)
Member of Technical Staff (Index Engineer, Search Crawler)

ResumeXrays LLC • Greater London

On-site
GBP 90,000 - 130,000
Senior Systems Engineer, Search Core & Indexing
Senior Systems Engineer, Search Core & Indexing

Perplexity • Greater London

On-site
GBP 90,000 - 130,000
Senior Data Systems Engineer - High-Performance AI Infra
Senior Data Systems Engineer - High-Performance AI Infra

present of AI • Greater London

Hybrid
GBP 90,000 - 130,000
Senior Data Engineer: Build Scalable Cloud Data Platforms
Senior Data Engineer: Build Scalable Cloud Data Platforms

Reply • Greater London

On-site
GBP 70,000 - 110,000
Data Engineer, AI Infrastructure & Search (Equity)
Data Engineer, AI Infrastructure & Search (Equity)

Solve Intelligence, Inc. • Greater London

Hybrid
GBP 100,000 - 160,000
Significant equity
Founding impact
Visa sponsorship
+2
Senior Search Core Engineer: High-Scale, Low-Level Systems
Senior Search Core Engineer: High-Scale, Low-Level Systems

Perplexity • Greater London

On-site
GBP 90,000 - 130,000