Senior Observability Engineer

Siri InfoSolutions Inc; Los Angeles (CA); On-site; USD 140,000 - 180,000; Full time

Job summary

Siri InfoSolutions Inc is seeking a Senior Observability Engineer in Woodland Hills, CA to elevate telemetry across infrastructure, apps, and business transactions. You will own dashboards, alerts, SLOs, and the automation that makes observability repeatable.

You will implement advanced baselining, anomaly detection, and CI/CD instrumentation gates, while scaling OpenTelemetry fleets and enabling GenAI-assisted triage for faster incident resolution.

Qualifications

• Seasoned observability expert with telemetry and monitoring leadership.
• Experience designing scalable, code-defined dashboards, alerts and SLOs.
• Strong skills in cloud telemetry across AWS/Azure/GCP and third-party SaaS.

Responsibilities

• Own, design and optimize alerting and noise reduction to improve signal quality.
• Implement dynamic baselining and anomaly detection tied to traffic patterns.
• Advance Observability-as-Code and CI/CD instrumentation gates in pipelines.
• Manage OpenTelemetry collectors and agent fleets for fleet health.
• Automate root cause analysis and correlate changes with incidents.
• Adopt GenAI/LLM capabilities for rapid log analysis and triage.
• Integrate telemetry across cloud services and data lakehouse pipelines.
• Drive end-to-end tracing of business transactions and define SLOs across services.
• Empower developers with self-service health dashboards and standards.

Skills

Observability

Telemetry architecture

Anomaly detection

SLO governance

Automation & IaC

Job description

Role: Senior Observability Engineer

Location: Woodland Hills, CA

Duration: Fulltime

Job Description

Must Have Technical/Functional Skills

Customer is seeking a seasoned Observability expert who doesn't just manage dashboards but actively lives and breathes telemetry architecture. In this role, Personnel will elevate customer observability maturity across infrastructure, applications, and business transactions.

Roles & Responsibilities

Personnel will own, design, and optimize the following core domains:

1. Operations & Noise Reduction

• Alert-to-Incident Signal Optimization: Analyze and optimize our Alert-to-Incident noise ratio (targeting a baseline better than 10:1). Drive the evolution from chaotic alerting to high-fidelity, actionable incident creation.
• Dynamic Baselining & Anomaly Detection: Shift the paradigm away from rigid static thresholds. Implement dynamic baseline that intelligently accounts for time-of-day, day-of-week, and seasonal traffic patterns.

2. Guardrails, Standards, & Observability-as-Code

• Observability-as-Code (OaC): Drive the maturity of our telemetry infrastructure by ensuring all dashboards, alerts, SLOs, and monitor configurations are defined, versioned, and deployed as code.
• CI/CD Instrumentation Gates: Establish and enforce automated instrumentation compliance gates within our deployment pipelines to ensure code is observable before it hits production.
• Fleet Health Management: Centrally manage, version, and monitor the health of our Open Telemetry (OTel) collectors and agent fleets.

3. Advanced Diagnostics & Next-Gen Tech

• Automated Root Cause Analysis (RCA): Implement platform capabilities that automatically surface probable root cause the moment an incident fire.
• Change & Deployment Correlation: Ensure all deployments, configuration changes, feature flag toggles, and database migrations are automatically annotated on dashboards and correlated to active incident timelines.
• GenAI/LLM-Assisted Triage: Evaluate and adopt GenAI/LLM capabilities for advanced log pattern explanation and accelerated incident troubleshooting.

4. Telemetry Architecture & Data Strategy

• Cloud-Native & Third-Party Monitoring: Ensure deep telemetry integration across cloud-managed services (AWS/Azure/GCP, EKS/AKS, Lambda, RDS) and critical third-party SaaS dependencies (e.g., Guidewire, Salesforce, Earnix, Uniphore, payment gateways).
• Lakehouse & Data Pipeline Integration: Architect pipelines to export raw telemetry data to our data Lakehouse (S3/ADLS) to power advanced ML pipelines and predictive analytics.
• Predictive Capacity Analytics: Leverage the observability platform for capacity forecasting-predicting utilization trends for CPU, memory, queue depth, and storage before saturation occurs.
• Log Standardization: Drive org-wide standards for log structure and serialization to ensure seamless cross-platform parsing and querying.

5. Culture, SLOs, & Business Impact

• End-to-End Business Transaction Tracing: Map and trac e complex, multi-service customer journeys (e.g., policy quote bind pay) to provide full-context business transaction visibility.
• SLO/SLA Governance: Define, implement, and track Service Level Objectives (SLOs) across all production services.
• Developer Empowerment & Self-Service: Democratize observability by fostering a proactive culture where developers instrument their own services during active development, backed by standardized, self-service health dashboards.