Senior Observability Engineer

Verda; Greater London; Remote; GBP 90,000 - 130,000; Full time

Benefits offered by this job

Cash and equity

Job summary

Verda is building a full-stack AI cloud with a global footprint, including London offices and remote EU teams. We seek a Senior Observability Engineer to shape an observability platform across bare-metal, VM, Linux, Kubernetes, and GPU AI workloads.

You’ll work with Go, Python, and Node.js services, implementing OpenTelemetry and AI-assisted monitoring. You’ll design scalable metrics, logs, and traces, automate deployments with Argo CD and GitLab CI/CD, and collaborate across SRE, infra,

Qualifications

• Strong hands-on experience with Grafana and production telemetry systems.
• Experience with distributed systems and Kubernetes in production.
• Practical experience with OpenTelemetry instrumentation and Collector pipelines.
• Experience with declarative deployments and CI/CD pipelines.
• Ability to explain technical decisions and guide teams.

Responsibilities

• Design, scale, and operate shared metrics, logs, and tracing services.
• Manage declarative Kubernetes deployments using Argo CD and GitLab CI/CD.
• Automate host and VM configuration with Ansible and SaltStack.
• Improve monitoring for bare-metal, Linux, networks, and Kubernetes.
• Build telemetry pipelines for GPU clusters and AI workloads.
• Define shared OpenTelemetry instrumentation standards and data collection.
• Build dashboards and alerts to improve incident response.
• Document standards and promote adoption of observability practices.

Skills

Observability platforms

Infrastructure

OpenTelemetry

GitOps

Operations

Cross-team collaboration

Tools

Grafana

VictoriaMetrics

VictoriaLogs

OpenTelemetry

Argo CD

GitLab CI/CD

Ansible

SaltStack

Job description

At Verda, we're building a full-stack AI cloud, covering everything from data centers and hardware to our own cloud platform that the world's leading AI teams use to do serious AI work.
We strive to make a positive mark on the world through the infrastructure we build and give leading teams a service they can truly depend on. Headquartered in Helsinki, we operate globally with offices in London and San Francisco.
Join Verda while it’s still being built - not once it’s finished.

About the role

As a Senior Observability Engineer on our Observability Platform team, you will contribute to the architecture and development of our observability platform across bare-metal servers, virtual machines, Linux, Kubernetes, GPU clusters, network infrastructure, and cloud-native applications.

You will help translate architectural decisions into practical implementations and shared standards. A core part of your work will involve operating and improving our observability stack built on Grafana and tools from the VictoriaMetrics and Prometheus ecosystems, improving incident detection and response, and developing AI-assisted monitoring capabilities.

Working closely with platform, SRE, infrastructure, network, security, application, and ML engineering teams, you will help integrate observability into how systems are designed and operated.

Your responsibilities

• Observability platform: Design, scale, and operate shared metrics, logs, and tracing services, covering infrastructure health and workload behavior.

• GitOps & automation: Manage declarative Kubernetes deployments using Argo CD and GitLab and Github CI/CD, and automate host and VM configuration using Ansible and SaltStack.

• Infrastructure telemetry: Improve monitoring for bare-metal hardware, Linux, physical and overlay networks, and Kubernetes. Integrate telemetry from sources such as IPMI/Redfish, PDUs, SNMP, gNMI, and NetFlow/sFlow.

• GPU & AI workloads: Build telemetry pipelines and dashboards for GPU clusters, AI training runs, and inference services.

• Application telemetry: Help define shared standards for OpenTelemetry instrumentation, context propagation, and Collector pipelines. Support teams instrumenting Go, Node.js, and Python services and APIs.

• Alerting & incident response: Build useful dashboards and alerts, improve notification workflows through PagerDuty, and participate in on-call rotations. Work with other teams to reduce alert noise and shorten incident diagnosis and recovery.

• Shared practices: Document standards, help teams adopt them, and evaluate and develop AI-assisted approaches to monitoring and incident investigation.

Your key competencies

• Observability platforms: Strong hands-on experience with Grafana and production telemetry systems, including VictoriaMetrics and VictoriaLogs or comparable metrics and logging backends.

• Infrastructure & distributed systems: Strong Linux fundamentals, production Kubernetes experience, and experience monitoring cloud-native applications and distributed systems.

• OpenTelemetry: Practical experience with instrumentation, metrics, traces, structured logs, trace context propagation, and Collector pipelines.

• GitOps & automation: Experience with declarative deployments, CI/CD, and configuration management using tools such as Argo CD, GitLab CI/CD, Ansible, and SaltStack.

• Operations: Experience investigating production incidents, maintaining alerts, and sharing operational responsibilities within an engineering team.

• Cross-team collaboration: Ability to explain technical decisions, guide teams, and support adoption of shared observability practices.

Nice to have

• Experience building and operating large-scale observability systems across multiple data centers.

• Experience monitoring GPU clusters and AI/ML workloads, including NVIDIA DCGM and training or inference performance metrics.

• Deeper experience with hardware and network telemetry, such as IPMI/Redfish, NetFlow/sFlow, SNMP, gNMI, or eBPF.

• Experience operating across bare-metal, VM, and Kubernetes environments.

• Experience applying LLMs, ML, or agent-based systems to automation and incident investigation.

• Exposure to security monitoring and collaboration with SecOps teams.

Why Verda

• Cash and equity compensation along with local benefits.

• 40+ nationalities, with 6 different ones on the management team.

• A real chance to make an impact and work alongside world class engineers, researchers, and partners across the global AI ecosystem.

Practicalities

• Location: London, UK or remote in EU

• Employment type: Full time and permanent