Senior Observability Engineer

Allegis Global Solutions (AGS); EE. UU.; A distancia; USD 140.000 - 200.000; Jornada completa

Descripción de la vacante

Allegis Global Solutions (AGS) is seeking a Senior Observability Engineer to lead our global observability platform. You will design and scale monitoring, logging, and tracing using Grafana, Prometheus, Loki, Tempo/Jaeger, OpenTelemetry, and IaC with Terraform.

Remote work from anywhere in the US is supported. The role requires 7+ years in Platform Engineering/SRE, strong Java or Python skills, and the ability to define SLI/SLOs, implement distributed tracing, and collaborate with

Formación

• 7+ years in Platform Engineering, SRE, or Observability.
• Hands-on with Grafana, Prometheus, Loki in production.
• Strong Terraform and IaC expertise.
• 2-3+ years of Java or Python development.
• Experience implementing OpenTelemetry observability.

Responsabilidades

• Design, implement, and scale enterprise observability solutions.
• Build dashboards, alerts, and visualization platforms.
• Define SLIs, SLOs, and error budgets.
• Improve alerting strategies and reduce noise.
• Partner with teams to drive observability adoption.

Conocimientos

Grafana

Prometheus

Loki

OpenTelemetry

Distributed Tracing

SLI / SLO / Error Budgets

Java or Python Programming

Monitoring & Alerting

CI/CD Integration

Herramientas

Terraform

Tempo/Jaeger/Zipkin

Kubernetes

Prometheus Exporters

Descripción del empleo

Job Title: Senior Observability Engineer (Grafana | Prometheus | Loki | Terraform | Open Telemetry)

Experience: 7-12 Years Employment Type: Full-Time Contract Work Mode: 100% Remote (Work from Anywhere)

Job Description

We are seeking an experienced Senior Observability Engineer to lead and scale our observability platform for a globally distributed engineering organization. The ideal candidate should have strong hands‑on expertise in Grafana, Prometheus, Loki, Terraform, and OpenTelemetry, along with 2-3 years of programming experience in Java or Python.

You will be responsible for designing and managing enterprise‑grade monitoring, logging, and distributed tracing solutions, enabling engineering teams to build highly reliable and observable systems.

Key Responsibilities

Observability Platform

• Design, implement, and manage enterprise observability solutions using Grafana, Prometheus, Loki, Tempo/Jaeger, and OpenTelemetry.
• Build and maintain monitoring dashboards, alerts, and visualization platforms.
• Define and manage SLIs, SLOs, and error budgets.
• Improve alerting strategies and reduce operational noise.
• Partner with engineering teams to drive observability adoption and best practices.

Infrastructure as Code (Terraform)

• Develop and maintain Terraform modules for observability infrastructure.
• Automate provisioning of monitoring, logging, and tracing resources.
• Manage Terraform state, workspace strategies, and CI/CD integrations.
• Collaborate with Platform and DevOps teams on IaC standards.

Instrumentation & Distributed Tracing

• Implement OpenTelemetry instrumentation across applications and services.
• Enable distributed tracing for microservices architectures.
• Standardize logging, metrics, and tracing practices.
• Build workflows correlating metrics, logs, and traces for faster troubleshooting.

Programming & Automation

• Develop automation tools, scripts, exporters, and integrations using Java or Python.
• Create custom monitoring solutions and Prometheus exporters.
• Automate dashboard provisioning, alert management, and reporting.

SRE & Incident Management

• Support incident response, RCA, and post‑mortem activities.
• Develop operational runbooks and monitoring playbooks.
• Drive reliability improvements through observability insights.

Mandatory Skills

• Grafana
• Prometheus
• Loki
• Terraform
• OpenTelemetry
• Distributed Tracing
• SLI / SLO / Error Budgets
• Java or Python Programming
• Monitoring & Alerting
• CI/CD Integration

Desired Skills

• VictoriaMetrics / Thanos / Cortex
• Grafana Tempo / Jaeger / Zipkin
• Kubernetes
• AWS, Azure, or GCP Monitoring Services
• PagerDuty / OpsGenie
• eBPF Observability Tools (Pixie, Cilium, Parca)

Candidate Profile

• 7+ years in Platform Engineering, SRE, DevOps, or Observability Engineering.
• Expert‑level experience with Grafana, Prometheus, and Loki in production environments.
• Strong Terraform and Infrastructure‑as‑Code expertise.
• 2-3+ years of Java or Python development experience.
• Experience implementing OpenTelemetry‑based observability.
• Strong communication and stakeholder management skills.
• Comfortable working in a fully remote environment.