Senior Principal Architect, Real-Time Data Platform

International Business Machines Corporation; Atlanta (GA); On-site; USD 180,000 - 240,000; Full time

Job summary

Confluent is seeking a Senior Principal Engineer to define and drive the multi-year technical strategy for the Background Plane within the Next-Gen Kafka architecture. This is an individual contributor role with executive-level scope spanning cloud storage, control planes, and reliability engineering.

You will lead initiatives across data references, encryption models, and task schedulers, mentoring senior engineers while aligning with global production rollout and architectural reviews.

Qualifications

• Proven track record architecting distributed systems at planetary scale.
• Deep insight into Apache Kafka internals or equivalent streaming systems.
• Ability to articulate and deliver multi-year architectural roadmaps.
• Fluent in reasoning about distributed concurrency and backpressure.

Responsibilities

• Define the multi-year technical strategy for the Background Plane and align roadmaps with business objectives.
• Design and optimize core platform capabilities including scheduling, coordination, autoscaling, and execution workflows.
• Ensure correctness and safety under extreme concurrency and failures across global infrastructure.
• Lead cross-organizational architectural contracts and drive production rollout.

Skills

Distributed Systems

Streaming & Storage

Strategic Technical Leadership

Rigorous Problem-Solving

Quality & SRE Rigor

Influential Communication

Mentorship / Leadership

Tools

Kafka Internals

Cloud Platforms AWS/GCP/Azure

Job description

Confluent is seeking a Senior Principal Engineer to define and drive the multi-year technical strategy for the Background Plane within the Next-Gen Kafka architecture. This is an individual contributor role with executive-level scope spanning cloud storage, control planes, and reliability engineering.

You will lead initiatives across data references, encryption models, and task schedulers, mentoring senior engineers while aligning with global production rollout and architectural reviews.