Senior Streaming Platform Engineer

Doist; San Francisco (CA); On-site; USD 155,000 - 400,000; Full time

Job summary

Sentry is building the Streaming Platform to power our ingestion pipelines and real-time data processing. You will design, build, and operate components including Kafka, the runtime, and developer-facing APIs to simplify deployment and running streaming applications at scale.

You will tackle challenges in distributed systems, real-time data processing, and developer experience, aiming for reliability, low latency, and reduced operational overhead across billions of events.

Qualifications

• 5+ years of software engineering experience with distributed systems or real-time streaming.
• Proficiency in Python or Rust; experience in Go or Java is valuable.
• Experience with Kafka, Flink, Spark Streaming or similar tools.
• Cloud experience with Kubernetes, AWS or GCP; strong delivery guarantees.

Responsibilities

• Design, build, and operate components of the Streaming Platform, including Kafka and the streaming runtime.
• Implement high-throughput stream processing with strong correctness guarantees.
• Build automation and a control plane for fleet management and observability.
• Collaborate with product engineers to enable fast, reliable ingestion pipelines.
• Improve observability and failover for mission-critical real-time systems.

Skills

Python

Rust

Go

Java

Distributed systems

Real-time streaming

Tools

Kafka

Flink

Spark Streaming

Kubernetes

AWS

GCP

ClickHouse

Arrow

Job description

Sentry is building the Streaming Platform to power our ingestion pipelines and real-time data processing. You will design, build, and operate components including Kafka, the runtime, and developer-facing APIs to simplify deployment and running streaming applications at scale.

You will tackle challenges in distributed systems, real-time data processing, and developer experience, aiming for reliability, low latency, and reduced operational overhead across billions of events.