Lead Distributed Tracing Architect for Observability

Socket.dev; Pleasanton (CA); Hybrid; USD 223,000 - 334,000; Full time

Job summary

Workday in Pleasanton, CA, seeks a senior engineer to own the architectural vision for distributed tracing within the Observability Platform. Design multi-petabyte pipelines using ClickHouse, Tempo, Kafka, Iceberg on AWS, delivering sub-second query performance and high availability.

You will mentor engineers, set direction across teams, and help shape Observability AI by feeding traces and logs into anomaly detection and automated incident management.

Qualifications

• 14+ years experience in software development engineering.
• 6+ years designing, building, and operating complex distributed systems.
• 8+ years experience with Java, Python or Go in production systems.

Responsibilities

• Architect and build Workday's distributed tracing platform on ClickHouse/Tempo for multi-petabyte scale and sub-second queries.
• Own the big-data pipeline: Kafka ingestion, Spark/Flink processing, Iceberg-on-S3 storage, including schema and lifecycle management.
• Drive performance, tuning, and HA/DR design across ingestion and query paths in production.
• Design security for multi-tenant access across ingestion and query layers.
• Lead operational excellence: monitoring, logging, alerting, capacity planning, on-call rotation.
• Shape Observability AI by enabling automated root-cause analysis and incident management.

Skills

Java

Python

Go

Education

Bachelor's degree in Computer Science or related field

Master's degree preferred or equivalent experience

Tools

ClickHouse

Tempo

Kafka

Spark/Flink

Iceberg

S3 / AWS

Job description

Workday in Pleasanton, CA, seeks a senior engineer to own the architectural vision for distributed tracing within the Observability Platform. Design multi-petabyte pipelines using ClickHouse, Tempo, Kafka, Iceberg on AWS, delivering sub-second query performance and high availability.

You will mentor engineers, set direction across teams, and help shape Observability AI by feeding traces and logs into anomaly detection and automated incident management.