Open-Source Data Query Engine Engineer

Palantir Technologies; Greater London; Hybrid; GBP 100,000 - 150,000; Full time

Benefits offered by this job

Hybrid work options

Job summary

Palantir Technologies, a world-changing software company, seeks a senior software engineer to design and optimize query planning, execution, and data processing for Foundry. You will extend engines, build Iceberg catalog integrations, and enable efficient reads and writes across large tables.

You will work with Spark, DataFusion, Iceberg, Arrow, and Parquet in a fast-paced, collaborative environment, delivering high-quality, scalable systems that power critical applications for hospitals, supply

Qualifications

• 4+ years of professional software engineering experience building production systems.
• Strong coding skills in Java, Rust, Scala, or C++.
• Experience with database engines, distributed data processing, or storage systems.

Responsibilities

• Designing and implementing query planning and optimization capabilities.
• Extending execution engines and improving operators, parallelism, and memory management.
• Developing Foundry’s Iceberg catalog and engine integrations and efficient reads/writes.
• Building shared transformation semantics and execution interfaces for multiple workloads.
• Improving incremental processing to reuse results while preserving correctness.
• Evaluating open-source data systems and validating performance on real workloads.
• Collaborating with product teams to translate needs into engine capabilities.

Skills

Java

Rust

Scala

C++

Education

Computer Science or related field

Tools

Apache Spark

Apache DataFusion

Apache Iceberg

Apache Arrow

Apache Parquet

Gradle

Cargo

GitHub

Job description

Palantir Technologies, a world-changing software company, seeks a senior software engineer to design and optimize query planning, execution, and data processing for Foundry. You will extend engines, build Iceberg catalog integrations, and enable efficient reads and writes across large tables.

You will work with Spark, DataFusion, Iceberg, Arrow, and Parquet in a fast-paced, collaborative environment, delivering high-quality, scalable systems that power critical applications for hospitals, supply