Principal Observability Engineer - Scale Telemetry Platform

Expedia Group; San Jose (CA); On-site; USD 249,000 - 348,500; Full time

Benefits offered by this job

Medical, dental, and vision coverage

Travel discounts

Wellness and travel reimbursement

Job summary

Expedia Group is seeking a Principal Software Engineer for Observability in San Jose. You will lead architecture and delivery of a centralized observability platform, handling logs, metrics and traces for thousands of services.

You will champion OpenTelemetry adoption, establish governance and cost controls, and unify tooling across teams while mentoring engineers and driving production reliability in a complex, cloud-native environment.

Qualifications

• Bachelor’s or Master’s degree in Computer Science or a related technical field, or equivalent practical experience.
• 10+ years of experience in software engineering, with a focus on building and operating large-scale distributed systems, infrastructure automation, or configuration management.
• Deep expertise in observability principles and the three pillars: logs, metrics, and traces.
• Strong hands-on proficiency with observability technologies such as Prometheus, Grafana, Datadog, Splunk, and OpenTelemetry.
• Proficiency in one or more of Go, Java, Python.
• Solid understanding of cloud-native architectures (Kubernetes, Docker, microservices) and major cloud platforms (AWS preferred).

Responsibilities

• Architect and build core telemetry pipelines: scalable pipelines for logs, metrics, traces and cost-effective scaling for 10x data growth.
• Drive OpenTelemetry adoption: strategy, rollout, and automated configurations for data collection across services.
• Implement platform governance and optimization: data governance, cost allocation, and SLOs for the platform; tooling for observability costs.
• Elevate observability practice: unify tooling (Grafana, Datadog, Splunk), documentation, and service lifecycle in the internal portal.
• Automate infrastructure lifecycle: IaC with Terraform and/or Crossplane; automate cluster provisioning and upgrades.
• Provide technical leadership and mentorship: architecture reviews and RFCs; mentor senior engineers.
• Production debugging: escalate complex incidents across the observability platform.
• Collaborate and innovate: work with Go, Java, Python, AWS, Kubernetes, OpenTelemetry, Prometheus, Grafana, Datadog, Splunk, Clickhouse.

Skills

Observability

OpenTelemetry

Prometheus

Grafana

Datadog

Splunk

Go

Java

Python

Kubernetes

Docker

AWS

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

OpenTelemetry

Prometheus

Grafana

Datadog

Splunk

Kubernetes

Docker

Clickhouse

Job description

Expedia Group is seeking a Principal Software Engineer for Observability in San Jose. You will lead architecture and delivery of a centralized observability platform, handling logs, metrics and traces for thousands of services.

You will champion OpenTelemetry adoption, establish governance and cost controls, and unify tooling across teams while mentoring engineers and driving production reliability in a complex, cloud-native environment.