Software Engineer, Knowledge Engine – Data Infrastructure

Exa

Zürich

On-site

CHF 140,000 - 180,000

Full time

10 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Exa is building a knowledge platform that synthesizes information from the web and beyond into one of the world’s largest open-domain knowledge graphs. In this role, you’ll construct knowledge pipelines that turn docs, datasets and APIs into ontology-shaped claims with provenance and confidence.

This greenfield work involves decisions on representation, trust, and evolving schemas so the graph remains correct as data changes while supporting idempotent and resumable processing at scale.

Qualifications

  • Experience building and operating large-scale distributed data pipelines in production.
  • Deep understanding of incremental processing, CDC, idempotent writes and backfills.
  • Strong Python skills and ability to read/tune Spark query plans.
  • Focus on correctness, reliability, and data quality at scale.

Responsibilities

  • Continuously synthesize live web crawl, vendor deliveries, APIs into a typed knowledge graph with provenance, timestamp and confidence.
  • Bring new sources in with versioned history and replayable cursors.
  • Decide what to trust via entity resolution and truth discovery across billions of records.
  • Keep the graph current with streaming changes, ensuring eventual consistency and resumable updates.
  • Measure precision, completeness and lineage to trace issues.

Skills

Python
Spark
Delta Lake
Iceberg
Hudi

Job description

The Knowledge Platform team builds Exa's knowledge engine. We continuously synthesize information from across the web and beyond into one of the world's largest open-domain knowledge graphs, with hundreds of billions of evidence-backed claims. The work has two halves: construction pipelines that decide what to trust, and retrieval engines that decide how best to search it. Our goal is to give agents answers to any factual question about the world with unprecedented precision and completeness.

In this role, you'll build the knowledge construction pipelines: turning everything from a 100k docs/s web crawl to vendor datasets into a graph of what is true about the world, and keeping it correct as data changes.

This is greenfield work with a massive scope. There is no off-the-shelf system for building a model of the world from the open web, so you'll make foundational decisions, from how claims are represented and resolved to how trust is assigned and how the graph evolves over time, that everything built on top of it depends on.

What you'll do
  • Continuously synthesize the live web crawl, vendor deliveries and APIs, and other off-the-web sources into one typed graph: extraction that turns pages and records into ontology-shaped claims, each carrying its provenance, timestamp and confidence.

  • Bring every new source in cleanly: versioned, append-only history with schema evolution, late-arriving and corrected records, and replayable cursors, so every claim traces back to the exact delivery that produced it.

  • Decide what to trust: entity resolution across billions of records (blocking, pairwise scoring, clustering) into canonical people, companies, places and publications, then truth discovery that weighs source reliability, recency and corroboration to reconcile conflicting claims.

  • Keep that picture current and convergent: streams arrive continuously, out of order and at very different rates, so the graph is eventually consistent by design. You'll build incremental computation over change data feeds, where each update is a diff of what changed and every job is idempotent and resumable, so the graph converges to the same state however runs interleave, fail or replay, and readers only ever see committed, consistent versions.

  • Measure precision and completeness: labeled evals for extraction and resolution, per-source coverage and correctness metrics, and lineage that pinpoints which source or stage broke and why.

What we're looking for
  • You've built and operated large-scale distributed data pipelines in production over petabytes of structured and unstructured data, with Spark over a lakehouse format (Delta Lake, Iceberg, Hudi).

  • You understand incremental processing deeply: change data capture, idempotent writes, backfills and reprocessing.

  • Strong in Python and at least one systems language; you can read and tune Spark query plans.

  • You obsess over correctness, reliability and data quality as much as scale.

Bonus
  • Entity resolution, record linkage, knowledge graphs or ontologies

  • Information extraction or truth discovery at scale

  • Lance or other columnar/vector-native formats

Example projects
  • Take a new billion-row dataset from first delivery to queryable entities in days, not months.

  • Keep a graph of hundreds of billions of claims current with incremental updates, so a fact that changes on the web reaches it in minutes.

  • Build a system that knows every company and person in the world, with one evidence-backed record each, resolved across the web and every source we ingest.

Exa is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, creed, color, religion, sex, sexual orientation, gender identity or expression, national origin, disability, age, veteran status, marital status, pregnancy or related conditions, criminal histories consistent with applicable law, or any other basis protected by applicable law.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Software Engineer, Knowledge Engine – Query
Software Engineer, Knowledge Engine – Query

Exa • Zürich

On-site
CHF 150,000 - 210,000
Premium healthcare
16 weeks parental leave
Monthly wellness stipend
Software Engineer, Knowledge Engine Data Infrastructure
Software Engineer, Knowledge Engine Data Infrastructure

present of AI • Zürich

On-site
CHF 130,000 - 180,000
Software Engineer, Knowledge Engine – Query Engine
Software Engineer, Knowledge Engine – Query Engine

Exa • Zürich

On-site
CHF 140,000 - 210,000
Premium healthcare benefits
Fertility benefits
16 weeks parental leave
+1
Research Engineer, Content Understanding
Research Engineer, Content Understanding

Speedrun Talent Network • Zürich

Hybrid
CHF 140,000 - 190,000
Data Infrastructure Engineer, Knowledge Graph & Pipelines
Data Infrastructure Engineer, Knowledge Graph & Pipelines

Exa • Zürich

On-site
CHF 140,000 - 180,000
Research Engineer, Content Understanding
Research Engineer, Content Understanding

Exa • Zürich

On-site
CHF 120,000 - 190,000
Research Engineer, Index Intelligence
Research Engineer, Index Intelligence

AItoolnavio • Zürich

Hybrid
CHF 110,000 - 170,000
Software Engineer, Knowledge Engine Query Engine
Software Engineer, Knowledge Engine Query Engine

present of AI • Zürich

Hybrid
CHF 120,000 - 190,000
Research Engineer, Index Intelligence
Research Engineer, Index Intelligence

Exa • Zürich

On-site
CHF 90,000 - 160,000
Senior Data Infra Engineer - Knowledge Graph Pipelines
Senior Data Infra Engineer - Knowledge Graph Pipelines

present of AI • Zürich

On-site
CHF 130,000 - 180,000
Top locations