Senior Principal Engineer, Kafka Platform

IBM; Lowell (MA); On-site; USD 180,000 - 240,000; Full time

Job summary

Confluent seeks a Senior Principal Engineer to drive the Background Plane strategy and architecture for real-time data streaming at scale. You will define roadmaps, own cross-team initiatives, and mentor the next generation of leaders to deliver highly available, secure systems.

You will shape cloud storage, scheduling, and consensus layers while ensuring end-to-end correctness and operational excellence across global infrastructure.

Qualifications

• Distributed systems at planetary scale with storage engines, log stores, or consensus protocols.
• Deep knowledge of Apache Kafka internals or equivalent streaming systems.
• Ability to define multi-year architectural roadmaps across teams.
• Strong reasoning about concurrency, race conditions, and backpressure.
• Proven ability to set SLA/SLO metrics and automated recovery.
• Excellent written and spoken communication for leadership and cross-team alignment.
• Mentorship passion for staff and principal engineers.

Responsibilities

• Architect multi-year strategic roadmap for the Background Plane and related architecture.
• Lead high-impact platforms including scheduling, coordination, autoscaling, and backpressure.
• Ensure correctness of data workflows with safety guarantees under failures.
• Scale to large partitions, throughput, and storage while maintaining sub-millisecond latency.
• Drive cross-organizational consensus on data references and tasks.
• Raise operational excellence with chaos testing and observability.
• Act as senior authority during architecture reviews and incident responses.
• Mentor senior engineers and represent vision externally.

Skills

Distributed Systems Mastery

Streaming & Storage

Strategic Technical Leadership

Rigorous Problem-Solving

Quality & SRE Rigor

Influential Communication

Talent Multiplier

Education

Master's Degree

Tools

Apache Kafka internals

Storage engines

Job description

Confluent seeks a Senior Principal Engineer to drive the Background Plane strategy and architecture for real-time data streaming at scale. You will define roadmaps, own cross-team initiatives, and mentor the next generation of leaders to deliver highly available, secure systems.

You will shape cloud storage, scheduling, and consensus layers while ensuring end-to-end correctness and operational excellence across global infrastructure.