Senior Software Engineer: Distributed Data & Open-Source

LanceDB Inc.; Oregon; Hybrid; USD 180,000 - 240,000; Full time

Job summary

LanceDB Inc. is looking for a Senior Software Engineer to expand Lance and LanceDB reach across data infrastructure. You’ll work at the intersection of high-performance computing, big data, and open-source systems, delivering scale, performance, and maintainability improvements.

You will drive integrations with Spark, Trino, and other data ecosystems, and contribute to open-source efforts while helping to promote the Lance format at conferences and in the community.

Qualifications

• 10+ years of experience in high-performance databases or large-scale data services.
• Deep understanding of open-source big data or AI training system internals (e.g., Spark, PyTorch, Trino).
• Strong experience with C++, Java, and/or Scala in HPC contexts.
• Experience with Rust or willingness to learn it.
• Ability to move fast, work independently, and collaborate with a top-tier team.

Responsibilities

• Design and maintain efficient distributed Lance dataset operations.
• Build indices to enable predicate pushdown and accelerate queries in Spark, Ray, or Trino.
• Work on table formats, data encodings, Lance format in Rust.
• Drive open-source community efforts to integrate Lance with Spark, Hive Metastore, Presto/Trino, Ray.
• Operate and improve internal data processing infrastructure.
• Promote the Lance format in open-source communities and at Big Data conferences.

Skills

High-performance databases

Big data systems

Open-source systems

C++/Java/Scala

Rust

Independent collaboration

Tools

Rust language

Job description

LanceDB Inc. is looking for a Senior Software Engineer to expand Lance and LanceDB reach across data infrastructure. You’ll work at the intersection of high-performance computing, big data, and open-source systems, delivering scale, performance, and maintainability improvements.

You will drive integrations with Spark, Trino, and other data ecosystems, and contribute to open-source efforts while helping to promote the Lance format at conferences and in the community.