Senior Observability Engineer for AI Infra Telemetry

Dada Consultants; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Dada Consultants is seeking a Senior Observability Engineer for a leading AI cloud infrastructure provider in Singapore. You will own the telemetry stack from hardware metrics collection to multi-tenant billing pipelines, shaping throughput and reliability.

Responsibilities include building scalable observability, integrating kernel-level tracing, and developing dashboards and alerting to detect degraded hardware before it impacts workloads.

Qualifications

• Bachelor's or master's degree in CS/EE or a related discipline.
• Strong experience with Prometheus/OpenTelemetry and Go in large-scale environments.
• Experience with Kubernetes metric exporters and operators, kernel tracing tools (eBPF/BCC).

Responsibilities

• Architect and scale a high-throughput observability platform across a large distributed compute environment.
• Integrate hardware telemetry (accelerator health, network fabric statistics) into a cluster-wide monitoring layer.
• Build kernel-level diagnostic tooling to trace network congestion, I/O latency, and bottlenecks.
• Develop dashboards and alerting pipelines to identify degraded hardware before workloads are affected.
• Design metering pipelines for real-time, multi-tenant usage-based billing.
• Partner with infra and scheduling teams to define observability standards for AI/ML workloads.
• Lead architecture reviews and mentor engineers on high-performance telemetry collection.

Skills

Prometheus

OpenTelemetry

Go

Kubernetes

eBPF/BCC

Linux performance

AI hardware metrics

Education

Bachelor's or Master's degree in Computer Science or Electrical Engineering

Tools

Kubernetes exporters

Kernel tracing tools

Linux performance tuning

Job description

Dada Consultants is seeking a Senior Observability Engineer for a leading AI cloud infrastructure provider in Singapore. You will own the telemetry stack from hardware metrics collection to multi-tenant billing pipelines, shaping throughput and reliability.

Responsibilities include building scalable observability, integrating kernel-level tracing, and developing dashboards and alerting to detect degraded hardware before it impacts workloads.