Remote Data Processing Engineer — Spark, Parquet, Rust/C++

DataPelago, Inc.; Mountain View (CA); Remote; USD 120,000 - 180,000; Full time

Job summary

DataPelago, Inc. is seeking a Data Processing Engineer - I/O to advance read/write capabilities of the DATA Pelago engine, focusing on high-throughput data flows in a modern distributed system.

You will shape interfaces with data sources and sinks, catalogs, and formats, and drive performance across large-scale data sets, ensuring efficiency and scalability in production environments.

Qualifications

• 7+ years in data processing or distributed systems.
• Deep experience with Spark/Parquet/ORC/Iceberg.
• Strong C/C++ with Rust preference.

Responsibilities

• Advance read/write capabilities of the DATA Pelago engine.
• Shape interfaces with data sources and sinks, catalogs, and formats; drive performance across large-scale data sets.

Skills

Spark

Parquet

ORC

Iceberg

C/C++

Rust

7+ years experience

Job description

DataPelago, Inc. is seeking a Data Processing Engineer - I/O to advance read/write capabilities of the DATA Pelago engine, focusing on high-throughput data flows in a modern distributed system.

You will shape interfaces with data sources and sinks, catalogs, and formats, and drive performance across large-scale data sets, ensuring efficiency and scalability in production environments.