Senior Software Engineer - Confluent Kafka

United States Digital Space LLC; Bengaluru; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Software in Bengaluru, India is seeking an experienced L3 Senior Software Engineer to help build Kora, a cloud-native event streaming platform. You’ll design, implement, and operate services powering connectors and Kafka/Flink artifacts in our Marketplace & Hub domain, delivering features with quality and reliability.

You will collaborate across teams, improve performance, scalability, and security, and participate in architectural reviews and on-call incident response.

Qualifications

• BS, MS, or PhD in computer science or a related field, or equivalent work experience.
• 4+ years of relevant cloud infrastructure/cloud networking experience.
• Strong fundamentals in distributed systems design and development.
• Experience building and operating large-scale systems in the Cloud.

Responsibilities

• Design, implement, and maintain services for connectors and Kafka/Flink artifacts in Marketplace & Hub.
• Deliver large features with tests, logging, and error handling.
• Build understanding of product area architecture to inform design choices.
• Collaborate with Kafka Connect, Security, Billing, UX on cross-team efforts.
• Improve performance, scalability, reliability, and security of services.
• Own on-call responsibilities and contribute to incident remediation.

Skills

Java/Scala/Go

Distributed systems

Cloud infrastructure

Production ops

JVM familiarity

Education

BS, MS, or PhD in computer science or related field

Tools

JVM

Job description

Introduction

At the company Software, we transform client challenges into solutions. Building the world's leading AI-powered, cloud-native products that shape the future of business and society. Our legacy of innovation creates endless opportunities for IBMers to learn, grow, and make an impact on a global scale. Working in Software means joining a team fueled by curiosity and collaboration. You'll work with diverse technologies, partners, and industries to design, develop, and deliver solutions that power digital transformation. With a culture that values innovation, growth, and continuous learning, the company Software places you at the heart of the company's product and technology landscape. Here, you'll have the tools and opportunities to advance your career while creating software that changes the world. With Confluent, data doesn't sit still. We put information in motion, streaming in near real time so organizations can react faster, build smarter, and deliver experiences as dynamic as the world around them.

Your Role And Responsibilities

About Product & Team

A software engineer on this team primarily contributes by implementing and deploying the business logic inside some KORA components that run in production. Their code defines how the system behaves, how features are delivered, and how the platform responds under real customer workloads.

KORA is the cloud native kafka engine. More about KORA https://www.confluent.io/blog/cloud-native-kafka-kora-vldb-award/

About The Role

We are a team of passionate engineers who love solving complex distributed systems and infrastructure problems. We are building Kora from the ground up to be a true managed service for clients who demand high levels of availability and performance at the lowest cost of ownership. In this role, you will be working on building highly available, reliable, secure, scalable, and multi-tenant Kora's compute layer. You will be instrumental in driving the technical solutions end-to-end, working closely with the team and key stakeholders to design, architect, and develop top-notch solutions. Here are examples of some of the challenges we've taken on in recent years:

Kora: A Cloud-Native Event Streaming Platform For Kafka

Apache Kafka's Next-Gen Rebalance Protocol: Towards More Stable and Scalable Consumer Groups

From On-Prem to Cloud-Native: Multi-Tenancy in Confluent Cloud

Speed, Scale, Storage: Our Journey from Apache Kafka to Performance in Confluent Cloud

Benchmarking Confluent Cloud

What You'll Do

As anL3 Senior Software Engineer, you are a mostly self-sufficient team member who owns and drives well-scoped to moderately complex problems end-to-end in your product area and helps other engineers with specific tasks when needed. In this role you will:

• Design, implement, and maintain services that power catalog, publishing, distribution, and lifecycle management for connectors and other Kafka/Flink artifacts, within a well-defined portion of the Marketplace & Hub domain.
• Deliver large features independently and with high quality, including tests, logging, and error handling, contributing to bigger initiatives owned by the team.
• Build a solid understanding of the design and architecture of your product area (e.g., specific Marketplace/Hub services, data models, and APIs) and use that to make sound design and implementation choices.
• Participate actively in technical processes for your team: design/one-pager reviews, RFC/KIP discussions where relevant, and PR reviews, providing constructive feedback to peers.
• Improve performance, scalability, reliability, and security of Marketplace & Hub services through incremental changes: refactors, dependency upgrades, schema evolution, and automation of existing manual workflows.
• Take shared ownership of operational excellence by handling on-call responsibilities, following runbooks, performing production operations safely, and contributing to better monitoring, alerting, and incident remediation.
• Collaborate with engineers across teams (Kafka Connect, Partner Engineering, Security, Billing, UX) on well-scoped cross-team efforts, unblocking yourself and others as needed.
• Help maintain a high engineering bar on the team through good coding practices, thoughtful reviews, and reliable delivery on commitments.

What You Will Do

• Driven and excited about challenges of delivering complex projects in production with a focus on quality
• Customer focused - making customers more successful by taking on their most challenging problems motivates you
• Have a strong sense of teamwork and able to make decisions which benefit the team and company
• Identify root causes, and get beyond treating symptoms - motivated to dig deep and solve hard problems

Preferred Education

Master's Degree

Required Technical And Professional Expertise

• BS, MS, or PhD in computer science or a related field, or equivalent work experience
• 4+ years of relevant cloud infrastructure/cloud networking experience
• Strong fundamentals in distributed systems design and development
• Experience building and operating large-scale systems in the Cloud
• Solid understanding of basic systems operations (disk, network, operating systems, etc)
• A self starter with the ability to work effectively in teams
• Proficiency in Java, Scala, Go or other statically typed languages

Preferred Technical And Professional Experience

• In-depth familiarity with the JVM
• A strong background in distributed storage systems or databases
• Experience in the areas of resource management and QoS
• Experience/knowledge with public clouds (AWS, Azure or GCP)
• Interest in evangelism (giving talks at tech conferences, writing blog posts evangelizing Kafka)
• Experience working on databases or data storage