Senior Principal Chaos Engineer, Platform & Resilience

Bybit

Hong Kong

On-site

HKD 1,200,000 - 2,000,000

Full time

14 days+

Application generator

Get a reply from this employer — a resume and cover letter tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Bybit is seeking a Senior Chaos/Resilience Engineer to design and operate an enterprise-grade chaos engineering platform across multi-cluster, multi-region deployments. You will own fault-injection tooling, safety controls, and automated recovery, partnering with SRE and platform teams to raise resilience across the production stack.

You will mentor 2–3 engineers, stay ahead of industry trends, and define playbooks for incident injection, monitoring, and SLO alignment.

Qualifications

8+ years backend/infrastructure engineering experience, with 3+ years in chaos or stability engineering.
Hands-on experience with large-scale fault injection in production environments.
Expert-level Kubernetes fault injection (Chaos Mesh / Litmus / custom solutions), familiar with CRD/Operator development.
Proficient in Go (or other backend language) with architecture design capability.
Deep understanding of distributed system failure modes.
Familiarity with observability stacks (Prometheus / Grafana / OpenTelemetry).
Excellent technical documentation and solution design skills.

Responsibilities

Design and build an enterprise-grade chaos engineering platform across multi-cluster, multi-region deployments.
Develop fault injection engine and safety controls for production environments.
Production safety: blast radius control, one-click Kill Switch, automated rollback, real-time impact monitoring.
Integrate with monitoring and SLO systems to automate fault observation and pass/fail decisions.
Mentor 2–3 engineers in chaos engineering and stay current with industry practices (AI-driven fault scenarios).
Collaborate with SREs and developers to improve system resilience across services.