Senior Software Engineer - Confluent - FlinkSQL

International Business Machines Corporation; Gemeindeverwaltungsverband Gärtringen/Ehningen; On-site; EUR 90,000 - 130,000; Full time

Job summary

IBM Software is seeking a Senior Software Developer to help build and evolve the Flink SQL engine at Confluent Cloud. You will focus on the SQL planner/optimizer, runtime operators, and integration with Kafka and Schema Registry, enabling powerful streaming and batch workloads in SQL.

You will collaborate with Flink runtime teams and contribute to reliability, observability, and on-call readiness. This role is based in Europe, with daily collaboration across Europe and North America.

Qualifications

• Solid understanding of SQL and relational concepts.
• Experience with backend services and distributed systems.
• Proficiency in Java (or JVM language) with a large codebase.
• Familiarity with Kafka, schema formats, and data pipelines.
• Cloud experience on AWS, GCP, or Azure; on-call ownership.
• Contributions to open-source streaming or distributed systems are a plus.

Responsibilities

• Implement and evolve the Flink SQL engine and planner.
• Improve streaming SQL operators and correctness from the SQL layer.
• Integrate SQL with Kafka topics, Schema Registry, and formats (Avro, Protobuf, JSON).
• Contribute to the Table API, UDFs, and INFORMATION_SCHEMA features.
• Improve observability with metrics, logs, and dashboards; participate in on-call rotation.

Skills

Java (preferred)

Distributed systems

Backend services

SQL / query engines

Kafka / Schema Registry

Cloud experience

Production ownership

Tools

Apache Kafka

Apache Flink

Schema Registry

Apache Calcite

Job description

Introduction

At IBM Software, we transform client challenges into solutions. Building the world’s leading AI-powered, cloud-native products that shape the future of business and society. Our legacy of innovation creates endless opportunities for IBMers to learn, grow, and make an impact on a global scale. Working in Software means joining a team fueled by curiosity and collaboration. You’ll work with diverse technologies, partners, and industries to design, develop, and deliver solutions that power digital transformation. With a culture that values innovation, growth, and continuous learning, IBM Software places you at the heart of IBM’s product and technology landscape. Here, you’ll have the tools and opportunities to advance your career while creating software that changes the world.

Your role and responsibilities

About the Role

The Stream Processing & Analytics (SPA) team is building an elastic, reliable, durable, cost effective, and performant stream processing engine for Confluent Cloud based on Apache Flink®. Stream processing is at the heart of innovations that unlock the next generation of use cases for Confluent Cloud.

As a Senior Software Developer on the FlinkSQL team, you will help build and evolve the SQL engine at the core of Confluent Cloud for Apache Flink® — the layer that makes stream processing feel as simple as using a database. You will work on the SQL planner and optimizer, SQL semantics and runtime operators, the catalog and metadata integration with Apache Kafka ® and Schema Registry, and the Table API, so that customers can express powerful streaming and batch workloads in SQL without worrying about the underlying infrastructure.

You will collaborate closely with other teams (for example, the Flink runtime and control-plane teams), but your primary focus will be on the SQL layer itself: usability, performance, and operability of Flink SQL on Confluent Cloud.

This role is based in Europe, where most of the FlinkSQL team is located. You will work daily with teammates across Europe and with partner teams in North America.

You will have the opportunity to collaborate closely with the Apache Flink Open Source community.

What You Will Do

Implement and improve features in the Flink SQL engine

• Contribute to the SQL parser, logical planner, and optimizer (based on Apache Calcite), including new SQL syntax, built-in functions, and planner rules.
• Improve the correctness and performance of streaming SQL operators (for example, joins, aggregations, and changelog processing) as seen from the SQL layer.

Make Flink SQL a great experience on Confluent Cloud

• Work on the integration between SQL and the Confluent ecosystem: how tables map to Kafka topics, Schema Registry schemas, data types, and formats (Avro, Protobuf, JSON).
• Contribute to features across the SQL surface, such as user-defined functions (UDFs), the Table API, and standard metadata interfaces like INFORMATION_SCHEMA.

Improve reliability and correctness of the SQL engine

• Deliver well-scoped improvements independently and with high quality, from clarifying requirements through implementation, testing, and rollout — with guidance from senior engineers on larger designs.
• Investigate and fix planner and semantic bugs, and extend our semantic test coverage so customer queries behave predictably.

Strengthen observability and operational excellence

• Add and refine metrics, structured logging, and dashboards that make SQL statements easier to understand and debug for customers and on-call engineers.
• Participate in the team's on-call rotation, supported by tested runbooks and processes, and contribute to incident follow-ups.
• Operate what you build in production.

Required technical and professional expertise

Experience and fundamentals

• 2-3+ years of industry experience building backend services, data infrastructure, or distributed systems.
• Strong computer science fundamentals, including data structures, algorithms, and a solid understanding of concurrency and distributed systems basics.
• Proficiency in Java (preferred for this role) or another JVM language (for example, Scala), with the ability to contribute to a large, existing codebase.

SQL and data processing

• A solid understanding of SQL and relational concepts, and genuine interest in how query engines work (parsing, planning, optimization, execution).
• Familiarity with data processing systems such as databases, query engines, or stream processors — through work experience, open source, or advanced coursework/projects.

Cloud and operations

• Experience running or operating services on at least one major public cloud (AWS, GCP, or Azure); exposure to Kubernetes-based environments is a plus.
• Willingness to own your code in production, including metrics, dashboards, alerts, and on-call participation.

Ways of working

• Ability to deliver well-scoped features end-to-end with high quality: clarify requirements, implement, test, and roll out safely, seeking input from senior engineers where needed.
• Clear, pragmatic communication and constructive participation in design discussions and code reviews.
• High bar for code quality and testing, and a willingness to iterate based on feedback.

Preferred technical and professional experience

• Experience with Apache Flink SQL or the Table API, or with another SQL engine or large-scale data processing system (for example, Spark SQL, Trino, or a database engine).
• Familiarity with query planners and optimizers, or compiler construction (Apache Calcite is a plus).
• Understanding of streaming semantics such as event time, watermarks, and CDC handling.
• Experience with Apache Kafka, Schema Registry, or serialization formats like Avro and Protobuf.
• Contributions to open-source projects in streaming, databases, or distributed systems, ideally including Flink or related components.

IBM is committed to creating a diverse environment and is proud to be an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, caste, genetics, pregnancy, disability, neurodivergence, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.