Senior Network Observability Engineer

clockworksystems; Palo Alto (CA); On-site; USD 140,000 - 210,000; Full time

Benefits offered by this job

Catered lunch

Competitive compensation

Great benefits package

Job summary

Clockwork Systems is hiring a Senior Software Engineer to architect, develop, and scale a high-performance network monitoring and observability platform. This role focuses on RDMA, RoCE, InfiniBand, and TCP/IP visibility, with strong emphasis on distributed systems and Linux networking.

You will build backend telemetry services, dashboards, alerts, diagnostics, and traffic analysis workflows, while solving production issues across software, OS, server, and network layers.

Qualifications

• Bachelor's or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or a related technical field.
• Strong hands-on programming experience in C++, Go, Python, Rust, or similar systems programming languages.
• Proven experience delivering complex infrastructure projects and contributing to the design and implementation of distributed systems.
• Experience building distributed systems, backend services, telemetry pipelines, or observability platforms.
• Hands-on experience with RDMA, RoCE, InfiniBand, or other high-performance network fabrics.
• Familiarity with libibverbs, RDMA verbs, RDMA CM, queue pairs, completion queues, memory registration, and related RDMA concepts.
• Strong knowledge of Linux networking, TCP/IP, DNS, HTTP, routing, MTU, congestion control, packet loss, latency, and performance tuning.
• Experience with traceroute-style diagnostics, path discovery, network reachability checks, synthetic probes, or active network measurements.
• Experience with monitoring and visualization platforms such as Prometheus, Grafana, Datadog, Splunk, OpenTelemetry, or similar tools.
• Strong debugging skills across software, operating system, server, and network layers.
• Experience operating production systems in Linux-based environments.
• Strong technical judgment and ability to build reliable, scalable, and maintainable systems.

Responsibilities

• Design, develop, and scale high-performance network monitoring platforms for RDMA, RoCE, InfiniBand, and TCP/IP infrastructure.
• Build backend telemetry services, observability dashboards, alerts, diagnostics, anomaly detection, SLA monitoring, and traffic analysis workflows.
• Troubleshoot complex production issues across application, OS, server, RDMA, and network layers while optimizing low-latency collection, aggregation, and alerting.
• Collaborate with cross-functional teams to deliver scalable solutions, improve engineering practices, automate operational workflows, and contribute to the technical direction of the platform.

Skills

C++

Go

Python

Rust

Education

Bachelor's or Master's degree in CS/CE/ECE

Tools

Prometheus

Grafana

Datadog

OpenTelemetry

Splunk

Job description

Clockwork Systems is hiring a Senior Software Engineer to architect, develop, and scale a high-performance network monitoring and observability platform. This role focuses on RDMA, RoCE, InfiniBand, and TCP/IP visibility, with strong emphasis on distributed systems and Linux networking.

You will build backend telemetry services, dashboards, alerts, diagnostics, and traffic analysis workflows, while solving production issues across software, OS, server, and network layers.