Senior Principal Engineer - Real-Time Data Systems Architect

International Business Machines Corporation; Austin (TX); On-site; USD 180,000 - 240,000; Full time

Job summary

Confluent is seeking a Senior Principal Engineer to define and drive the multi-year technical strategy for the Background Plane within the Next-Gen Kafka architecture. This individual-contributor role spans cloud storage, control planes, and SRE, mentoring staff while aligning leaders on mission-critical decisions.

You will lead architectural initiatives across cloud infrastructure, consensus, and site reliability engineering, with executive-level scope and broad impact.

Qualifications

• Experience architecting and operating mission-critical distributed systems at planetary scale.
• Deep expertise in storage engines, distributed logs, replication, or consensus.
• Strong ability to translate business goals into multi-year roadmaps.
• Proven leadership across multi-disciplinary engineering teams.
• Excellent communication for executive stakeholders.

Responsibilities

• Architect multi-year strategy for the Background Plane and drive roadmaps.
• Design and optimize high-scale processing platforms for scheduling and coordination.
• Ensure correctness across data workflows including compaction, rewriting, retention.
• Scale to large partition counts and sub-millisecond foreground latencies.
• Lead architectural consensus across orgs and manage production rollout.
• Define engineering standards for fault tolerance and observability.
• Act as senior technical authority during reviews and incidents.
• Mentor senior engineers and represent the company publicly.

Skills

Distributed systems

Kafka internals

Strategic leadership

Problem solving

SRE discipline

Communication skills

Mentorship

Job description

Confluent is seeking a Senior Principal Engineer to define and drive the multi-year technical strategy for the Background Plane within the Next-Gen Kafka architecture. This individual-contributor role spans cloud storage, control planes, and SRE, mentoring staff while aligning leaders on mission-critical decisions.

You will lead architectural initiatives across cloud infrastructure, consensus, and site reliability engineering, with executive-level scope and broad impact.