We are looking for a highly experienced Staff/Principal Backend Engineer to own the scalability, reliability, and architecture of the entire platform across workflows, conversations, and ticketing systems. This is a senior technical IC role requiring strong hands-on experience with distributed systems, large-scale production infrastructure, failure handling, and cross-service architecture. You will work across engineering teams to identify bottlenecks, prevent production failures, and build systems that can scale reliably.
Responsibilities
- Own the scalability and reliability of the platform across workflows, conversations, ticketing, and supporting services.
- Identify and proactively address system bottlenecks, failure points, capacity risks, and reliability gaps.
- Drive cross-team architecture and technical decisions across multiple services and engineering teams.
- Design and evolve highly scalable, fault-tolerant distributed systems.
- Personally own and troubleshoot large-scale production systems, including critical incidents and outages.
- Work extensively on queues and asynchronous processing, backpressure and load management, Redis reliability and failure modes, Kubernetes scaling and orchestration, and multi-database architectures.
- Service-to-service communication and distributed workflows.
- Lead production incident investigations, root-cause analysis, and long-term reliability improvements.
- Define and improve engineering practices around capacity planning, observability, performance, resilience, and disaster recovery.
- Collaborate with engineering teams to ensure systems are designed for high availability, scalability, and operational excellence.
- Remain hands-on and contribute approximately 25-33% of your time to coding, particularly on the most technically challenging problems.
Requirements
- 10+ years of backend/software engineering experience with significant experience building and operating large-scale production systems.
- Strong experience as a staff engineer, principal engineer, distributed systems engineer, or hands-on platform architect.
- Proven ownership of large-scale production systems through outages and high-severity incidents.
- Deep understanding of distributed systems, scalability, and high availability; fault tolerance and resilience, queuing systems and backpressure, caching and Redis, Kubernetes and container orchestration, Multi-database architectures; capacity planning, and performance optimization
- Strong programming experience in Node.js and/or Go is required.
- Ability to make architecture-level decisions across multiple teams and services.
- Strong debugging and problem-solving skills, with the ability to get hands-on when systems fail.
- Experience balancing architecture, reliability, performance, and business requirements.
- Strong communication skills and ability to influence senior engineers and technical stakeholders without direct authority.
- Ideal Candidate: The ideal candidate is a hands-on staff/principal backend engineer who has built and operated distributed systems at scale, has personally dealt with production failures and outages, and can drive architecture across multiple engineering teams.