Staff Software Engineer, Kafka & Distributed Storage

International Business Machines Corporation; Tucson (AZ); On-site; USD 150,000 - 190,000; Full time

Job summary

IBM Software is driving the next generation of AI-powered, cloud-native storage and analytic capabilities. As part of the Kora project, you will advance the storage and query stack, enabling streaming data to be consumed as both a table and a stream, with a high-performance analytical engine.

You will tackle multi-tenant, scalable compute infrastructure and ensure reliability across Confluent Cloud components.

Qualifications

• BS, MS, or PhD in computer science or a related field, or equivalent work experience.
• Excellent communication and collaboration skills, with the ability to influence and guide stakeholders at all levels.
• Experience with multi-tenant, highly available compute infrastructure, and the core components for table materialization and background maintenance.

Responsibilities

• Design and implement storage engine features, including table schematization, materialization, and maintenance at scale.
• Build and optimize a fast, scalable query engine for analytical workloads.
• Develop platform infrastructure powering background tasks for long-term storage and query serving with high availability and reliability.
• Collaborate with product, design and other engineering teams to integrate storage and query features across the system.

Skills

Distributed storage

Query engine development

Multi-tenant infrastructure

Cross-functional collaboration

Education

Master's Degree

Tools

Apache Kafka

Apache Iceberg

Apache Flink

Apache Fluss

Presto/Trino

Spark SQL

DuckDB

ClickHouse

AWS / Azure / GCP

Job description

IBM Software is driving the next generation of AI-powered, cloud-native storage and analytic capabilities. As part of the Kora project, you will advance the storage and query stack, enabling streaming data to be consumed as both a table and a stream, with a high-performance analytical engine.

You will tackle multi-tenant, scalable compute infrastructure and ensure reliability across Confluent Cloud components.