Staff Software Engineer, Replication Foundations

Openkrill; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Temporal Technologies is hiring a Staff Software Engineer to join the Replication Foundations team within Temporal’s Cloud Global Services (CGS) organization.

Replication Foundations owns and evolves Temporal’s core replication stack, the backbone behind High Availability namespaces, cross-cluster and cross-region replication, and migrations between self-hosted and cloud deployments.

Qualifications

• Proven track record delivering production distributed systems.
• Ability to define system architecture and protocol behavior.
• Experience debugging complex production issues including concurrency and performance.

Responsibilities

• Set the technical direction for Temporal’s OSS replication stack.
• Lead design and implementation of replication protocols for HA and cross-cluster scenarios.
• Drive scalability and reliability initiatives across multiple namespaces and clusters.
• Define system-level guarantees: consistency, ordering, and failure recovery.
• Mentor engineers and participate in design reviews and incident response.

Skills

Distributed systems

Go programming

System architecture

Debug production issues

Cross-team collaboration

Tools

Go toolchain

Job description

ROLE SUMMARY

We’re hiring a Staff Software Engineer to join the Replication Foundations team within Temporal’s Cloud Global Services (CGS) organization.

Replication Foundations owns and evolves Temporal’s core replication stack in Temporal OSS—the distributed systems backbone behind key Temporal Cloud capabilities such as High Availability namespaces, cross-cluster and cross-region failover, and migration products that enable customers to move workloads between self-hosted Temporal and Temporal Cloud. The team also builds foundational scalability and reliability mechanisms that support Temporal at scale.

In this role, you’ll help set the technical direction for Temporal’s distributed replication systems. You’ll lead complex, correctness‑critical initiatives spanning architecture, design, implementation, rollout, and operations. You’ll work across teams to evolve reliable and scalable replication capabilities that support both the open source project and Temporal Cloud.

WHAT YOU'LL DO

• Set the technical direction and evolve the architecture of Temporal’s OSS replication stack, from problem definition through rollout and operational support.
• Lead the design and implementation of replication protocols that power:
  • High Availability namespaces
  • Cross-cluster and cross-region replication
  • Migration between Temporal clusters, including cloud-to-self‑hosted and cloud-to‑cloud scenarios
• Drive scalability and reliability initiatives such as:
  • Multi-cell namespaces
  • Enabling a namespace to span multiple clusters
  • Improving load distribution and handling hot spots
• Define and communicate system‑level guarantees, including consistency models, ordering, idempotency, failure recovery, performance, and operational behavior.
• Identify architectural risks and opportunities, and shape the technical roadmap for replication capabilities that support current and future cloud products.
• Partner with Cloud Enablement, CGS, Product, and other engineering teams to align OSS replication foundations with customer and product needs.
• Lead design reviews, raise the quality of implementation and testing practices, mentor engineers, and provide technical guidance across the organization.
• Lead or contribute to debugging complex production issues, incident response, and follow‑up improvements related to replication and core system behavior.

WHAT YOU'LL BRING

• A track record of designing and delivering complex production distributed systems, including systems where correctness, availability, and performance are critical.
• Deep understanding of distributed systems fundamentals such as replication, partitioning, consistency, fault tolerance, durability, concurrency, and failure recovery.
• Experience defining system architecture, protocol behavior, invariants, and trade‑offs in ambiguous or evolving problem spaces.
• Experience debugging complex production issues, including concurrency bugs, data inconsistencies, partial failures, and performance bottlenecks.
• Proficiency writing production‑quality concurrent code in Go; experience with Java, C++, or similar systems languages is also welcome.
• Strong written and verbal communication skills, including the ability to explain complex designs and trade‑offs to both technical and cross‑functional audiences.
• Demonstrated ability to influence technical direction across teams, build alignment without direct authority, and mentor engineers.
• A thoughtful and curious approach to understanding how systems behave under load, failure, and changing workload conditions.

NICE TO HAVE

• Experience designing or maintaining replication protocols or data‑plane infrastructure.
• Experience with multi‑cluster or multi‑region architectures, including active‑active or active‑passive systems.
• Familiarity with database internals, log‑based replication, or event‑sourced systems.
• Prior contributions to large open source projects or distributed systems infrastructure.

Temporal Technologies is an Equal Opportunity Employer. Temporal Technologies does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non‑disqualifying physical or mental disability, national origin, veteran status, or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need. We embrace and celebrate differences and diversity.

Temporal is committed to providing access, equal opportunity, and reasonable accommodation for individuals with disabilities in employment, its services, programs, and activities. If you need to request a reasonable accommodation, please let your Recruiter know so we can assist.