Senior Data Systems Engineer – Real-Time Pipelines

Unity Technologies SF; Mountain View (CA); On-site; USD 210,000 - 273,000; Full time

Job summary

Unity Technologies SF is seeking a Senior Software Engineer to architect and implement distributed data systems at scale for near real-time reporting. You will design high-throughput, low-latency pipelines and operate in a cloud-native environment, collaborating across ML, Reporting, and external partners to deliver accurate signals.

The role emphasizes architectural modernization, including migrating legacy event handling to a modern gRPC-based service, and ownership of ad-serving

Qualifications

• Strong foundation in distributed systems and systems design.
• Hands-on experience building and operating large-scale data processing systems.
• Deep understanding of streaming concepts: exactly-once semantics, watermarking and event-time processing, stateful stream processing, checkpointing and recovery, and backpressure handling.
• Production experience with frameworks such as Apache Flink, Spark, Kafka, or similar technologies.
• Proficiency in Python, Java, or Scala.
• Experience with workflow orchestration tools such as Airflow for stream and batch coordination.
• Strong understanding of cloud-native architectures and distributed infrastructure including Kubernetes, containerization, and cloud platforms.

Responsibilities

• Own the end-to-end attribution pipeline and be a core engineer on data signals from ad interaction to data warehouse and ML/Reporting teams.
• Build and operate real-time ad event processing for terabytes of data at scale including enrichments, transformations, and privacy operations.
• Lead architectural modernization with migration to a gRPC-based service, ensuring backfill and failover without data loss.
• Improve low-latency data services and caching/persistence across ad lifecycle.
• Build and maintain ad serving infrastructure including caching, logging, tracing, authentication, and Kafka-backed tooling.
• Drive real-time data health monitoring and batch pipelines producing daily BigQuery datasets.
• Collaborate with ML, Reporting, and external partners for timely signal reporting and recovery efforts.

Skills

Distributed systems

Systems design

Streaming concepts

Python/Java/Scala

Airflow

Kubernetes

Cloud-native architectures

Tools

Apache Flink

Spark

Kafka

Python

Java

Scala

Airflow

Job description

Unity Technologies SF is seeking a Senior Software Engineer to architect and implement distributed data systems at scale for near real-time reporting. You will design high-throughput, low-latency pipelines and operate in a cloud-native environment, collaborating across ML, Reporting, and external partners to deliver accurate signals.

The role emphasizes architectural modernization, including migrating legacy event handling to a modern gRPC-based service, and ownership of ad-serving