Senior Data Engineer — Real-Time Pipelines (Equity)

SpaceXSI; Palo Alto (CA); Presencial; USD 180.000 - 330.000; Tempo integral

Benefícios oferecidos por esta vaga

Equity

Medical/Vision/Dental

401(k) retirement plan

Resumo da vaga

SpaceXSI is building a distributed data platform powering billions of events daily and providing data for internal teams. You will design, build, and operate real-time and batch pipelines, creating datasets and data products used by product, growth, safety, and business teams.

You will move across engines like BigQuery, Trino, Clickhouse, Flink, Kafka, and Spark, ensuring data correctness and enabling self-serve access to data. A strong track record in distributed data systems is essential.

Qualificações

• 5+ years of professional software engineering experience.
• Experience with Python, Rust, Scala, Go or Java, and data pipeline toolings and distributed systems.

Responsabilidades

• Design, build, and operate production‑grade realtime and batch pipelines that ingest, process, validate, and deliver data powering user‑behavior insights and product decisions.
• Create shared datasets, fact tables, and internal data products that let other teams analyze, debug, and improve product performance.
• Prototype and build tooling that automates and accelerates internal data workflows — backfills, dashboards, report generation, and self‑serve access to data.
• Own data correctness end to end: validate with output invariants, denominator reconciliation, and independent recomputation, and lead root‑cause investigations when key metrics move unexpectedly.
• Move fluidly across query engines and frameworks (e.g., BigQuery, Trino, Clickhouse for analytics; Flink, Kafka, Spark/Scalding for streaming and batch), choosing the right tool and adapting quickly to new infrastructure and environments.
• Partner across product and business teams to surface where data gaps exist and prioritize the highest‑impact opportunities for new data acquisition and improvement.
• Iterate quickly on feedback, shipping the smallest useful increment with a strong bias toward efficient, accurate, and reliable solutions.

Conhecimentos

5+ years of professional software eng.

Distributed systems

Ferramentas

Python

Rust

Scala

Go

Java

BigQuery

Trino

Clickhouse

Spark

Kafka

Flink

Scalding

SQL

Descrição da vaga

SpaceXSI is building a distributed data platform powering billions of events daily and providing data for internal teams. You will design, build, and operate real-time and batch pipelines, creating datasets and data products used by product, growth, safety, and business teams.

You will move across engines like BigQuery, Trino, Clickhouse, Flink, Kafka, and Spark, ensuring data correctness and enabling self-serve access to data. A strong track record in distributed data systems is essential.