Data Engineer — Real-Time Analytics & Pipelines

SpaceXSI; Palo Alto (CA); On-site; USD 125,000 - 400,000; Full time

Job summary

SpaceXSI in Palo Alto seeks a Data Engineer to join the X product engineering team. You will help build and operate a distributed data platform powering real-time and batch pipelines handling billions of events daily for multiple internal stakeholders.

You will own data correctness end to end, choose the right tools across Spark, Kafka, Flink and SQL, and partner with product, engineering, legal and finance to surface data gaps and drive high‑impact improvements.

Qualifications

• Hands‑on software engineering experience (3+ years) in data engineering or distributed systems.
• Proficient in one of Python, Rust, Scala, Go or Java, and familiar with data pipeline tooling.
• Experience solving large scale problems and delivering incremental quality improvements.

Responsibilities

• Design, build, and operate production‑grade realtime and batch pipelines ingesting and processing data.
• Create datasets and internal data products for analysis and product improvement.
• Prototype tooling to automate data workflows, backfills, dashboards and self‑serve data access.
• Own data correctness end to end: invariants, reconciliation, and root‑cause investigations.
• Work across query engines and frameworks (BigQuery, Trino, Clickhouse; Flink, Kafka, Spark) as needed.
• Collaborate with product and business teams to surface data gaps and prioritize improvements.
• Iterate rapidly, shipping small but useful increments with high reliability.

Skills

Python

Rust

Scala

Go

Java

Tools

Spark

Kafka

Flink

SQL

Job description

SpaceXSI in Palo Alto seeks a Data Engineer to join the X product engineering team. You will help build and operate a distributed data platform powering real-time and batch pipelines handling billions of events daily for multiple internal stakeholders.

You will own data correctness end to end, choose the right tools across Spark, Kafka, Flink and SQL, and partner with product, engineering, legal and finance to surface data gaps and drive high‑impact improvements.