Observability Engineer

Sepal; Remote; USD 80,000 - 100,000; Part time

Job summary

A technology company specializing in AI is seeking an Observability Engineer to design and optimize complex queries over large telemetry datasets. The ideal candidate will have 3+ years of experience in observability or production engineering, strong SQL skills, and a deep understanding of tools like Clickhouse and Grafana. This role offers remote work, flexible hours, and competitive pay of $50 to $85 per hour, depending on experience.

Qualifications

• 3+ years of experience in observability engineer, production engineer, or platform engineer roles.
• Strong SQL skills with a focus on performance problems and query optimization.
• Deep expertise in Clickhouse and Grafana, with a focus on large-scale query optimization.

Responsibilities

• Design complex, distributed queries over massive log and telemetry datasets.
• Create and manage synthetic datasets simulating real‑world logs.
• Explore ways to challenge AI's reasoning and log analysis skills.

Skills

SQL optimization

Clickhouse

Grafana

Distributed systems

Tools

Clickhouse

Grafana

Job description

Overview

The Observability Engineer / Production Engineer will design and optimize complex queries over large telemetry datasets to support AI‑driven incident investigation and system reliability. This role involves creating synthetic datasets simulating cloud infrastructure logs and working extensively with tools like Clickhouse and Grafana. Candidates should have 3+ years of experience in observability or production engineering with strong SQL and query optimization skills.

About Sepal AI

Sepal AI builds the world’s hardest tests for AI grounded in real‑world software systems. We are looking for an Observability Engineer with 3+ years of experience to help us understand, debug, and operate complex production systems at scale. You will work deeply with production logs, metrics, and traces, building tasks and datasets that can teach AI to investigate incidents, analyze performance, and improve system reliability.

What You’ll Do

• Design complex, distributed queries over massive log and telemetry datasets.
• Explore creative ways to challenge AI's reasoning ability and log analysis skills.
• Create and manage synthetic datasets that simulate real‑world DevOps, observability, or cloud infrastructure logs.

Who You Are

• 3+ years of experience in observability engineer, production engineer, or platform engineer roles.
• Deep expertise in Clickhouse and Grafana, with a focus on large‑scale query optimization, schema design, and performance tuning.
• Strong SQL skills: you know how to reason through performance problems and spot inefficient query patterns.

Compensation & Logistics

Pay: $50 – 85/hr depending on experience. Remote, flexible hours. Project timeline: 5–6 weeks.

Keywords

Observability Engineer, Production Engineer, Clickhouse, Grafana, SQL optimization, Distributed systems, Telemetry data, Cloud infrastructure logs, AI debugging, System reliability.