Confluent - Senior Software Engineer II - Kafka

International Business Machines Corporation; Gemeindeverwaltungsverband Gärtringen/Ehningen; On-site; EUR 90,000 - 130,000; Full time

Job summary

International Business Machines Corporation in Germany is seeking a Senior Software Engineer to advance the Realtime Context Engine, focusing on multi-tenant, highly available compute infrastructure and the query execution core.

You will work with product management, design, and other engineering teams to deliver scalable storage features and infrastructure, ensuring performance, reliability, and seamless integration across Confluent Cloud.

Qualifications

• BS/MS/PhD in computer science or a related field.
• 3–5 years of work experience building database technologies.
• Excellent communication and collaboration skills, with the ability to influence and guide stakeholders at all levels.
• Experience in building and operating large-scale database systems.

Responsibilities

• Query Engine Development: Design and implement innovative strategies for query execution, planning, and caching at massive scale.
• Platform Engineering: Build and maintain multi-tenant, highly available infrastructure that powers computational tasks for Lightning tables and the realtime context engine.
• Cross-Functional Collaboration: Collaborate with product management, design, and other engineering teams to ensure seamless integration of storage features and infrastructure with the broader organization.

Skills

C++

Rust

Large-scale databases

Communication

Collaboration

Education

BS/MS/PhD in Computer Science or related field

Tools

Apache Iceberg

Velox

Substrait

Job description

Introduction

At IBM Software, we transform client challenges into solutions. Building the world’s leading AI-powered, cloud-native products that shape the future of business and society. Our legacy of innovation creates endless opportunities for IBMers to learn, grow, and make an impact on a global scale. Working in Software means joining a team fueled by curiosity and collaboration. You’ll work with diverse technologies, partners, and industries to design, develop, and deliver solutions that power digital transformation. With a culture that values innovation, growth, and continuous learning, IBM Software places you at the heart of IBM’s product and technology landscape. Here, you’ll have the tools and opportunities to advance your career while creating software that changes the world.

Your role and responsibilities

As a Senior Software Engineer on the team, you will focus on advancing key layers of the Realtime Context Engine project, including its multi-tenant, highly available compute infrastructure, and the core components responsible for query execution and background materialization. You will tackle challenges such as developing strategies for efficient execution and at massive scale across all of Confluent Cloud, ensuring high availability and scalability.

What You Will Do:

Query Engine Development: Design and implement innovative strategies for query execution, planning, and caching at massive scale.

Platform Engineering: Build and maintain multi-tenant, highly available infrastructure that powers computational tasks for Lightning tables and the realtime context engine. Ensure that the platform is optimized for scalability, reliability, and resilience, in alignment with organizational objectives.

Cross-Functional Collaboration: Collaborate with product management, design, and other engineering teams to ensure seamless integration of storage features and infrastructure with the broader organization.

Required technical and professional expertise

• BS, MS, or PhD in computer science or a related field, or equivalent work experience




• 3-5 years work experience building database technologies




• Excellent communication and collaboration skills, with the ability to influence and guide stakeholders at all levels.




• Experience in building and operating large-scale database systems.

Proficiency in C++ or Rust

Preferred technical and professional experience

• Familiarity with or has experience contributing to the following open-source technologies: Apache Iceberg, Velox, Substrait




• Experience/knowledge with public clouds (AWS, Azure or GCP)

IBM is committed to creating a diverse environment and is proud to be an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, caste, genetics, pregnancy, disability, neurodivergence, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.