Data Engineer – Real-Time Pipelines & Distributed Systems

Xai; Palo Alto (CA); On-site; USD 125,000 - 400,000; Full time

Job summary

SpaceXAI is seeking a Data Engineer to join the product engineering team. You will design and operate distributed data pipelines supporting billions of events daily while delivering data products and datasets for internal teams and end-users.

The role requires hands-on coding and strong collaboration across Product, Algorithm, Legal, Finance and Sales. You will work with modern data tooling including Spark, Kafka, Flink, BigQuery and more, owning data correctness and driving data-driven product

Qualifications

• 3+ years of professional software engineering experience, ideally in data engineering or distributed systems.
• Hands-on expertise in Python, Rust, Scala, Go or Java, and data pipeline toolings and distributed systems.
• Knowledge of realtime and batch data processing tools such as Spark/Kafka/Flink/SQL and various storage systems in RMDBs/NoSQL.
• Experience interpreting product requirements into engineering plans and communicating with AI, product, marketing/sales and engineering teams.

Responsibilities

• Design, build, and operate production-grade realtime and batch pipelines ingesting data for user insights and product decisions.
• Create shared datasets, fact tables, and internal data products for analysis and debugging.
• Prototype tooling to automate internal data workflows, including backfills, dashboards, and self-serve data access.
• Own data correctness end-to-end with invariants, reconciliation, and root-cause investigations.
• Move across query engines and frameworks (BigQuery, Trino, ClickHouse; Flink, Kafka, Spark/Scalding) selecting appropriate tools.
• Collaborate with product and business teams to surface data gaps and prioritize high-impact data acquisitions.
• Iterate quickly with smallest useful increment and a bias toward reliable solutions.

Skills

Python

Rust

Scala

Go

Java

Data pipelines

Distributed systems

Spark

Kafka

Flink

SQL

BigQuery

Trino

ClickHouse

Scalding

Tools

BigQuery

Trino

ClickHouse

Spark

Kafka

Flink

Scalding

SQL

Job description

SpaceXAI is seeking a Data Engineer to join the product engineering team. You will design and operate distributed data pipelines supporting billions of events daily while delivering data products and datasets for internal teams and end-users.

The role requires hands-on coding and strong collaboration across Product, Algorithm, Legal, Finance and Sales. You will work with modern data tooling including Spark, Kafka, Flink, BigQuery and more, owning data correctness and driving data-driven product