Confluent - Distributed Systems Software Engineer - WarpStream

International Business Machines Corporation; San Diego (CA); Remote; USD 140,000 - 190,000; Full time

Job summary

IBM is seeking a Senior Engineer to join WarpStream, a Kafka-compatible data streaming platform built on object storage. You will own end-to-end features from the storage engine to the developer console, shaping scalable, multi-region services across cloud providers.

You will work in a fully remote, globally distributed team, contributing to high-availability backends and operational excellence while mentoring others.

Qualifications

• 5+ years industry experience designing, building, scaling and supporting backend systems in production
• Strong programming and algorithmic skills in Java, Go, C/C++, Python
• Deep curiosity about distributed systems and storage systems
• Focus on project delivery and communication
• Experience in operational excellence for large production services
• Customer centricity, teamwork, leadership and mentorship

Responsibilities

• Independently drive execution of complex technical projects end to end
• Build mission-critical backend services that deliver value to customers including architecture, design, development and operations
• Troubleshoot and debug issues inside a deep stack with microservices, containers, and virtualization
• Ensure operational readiness and meet availability and performance SLAs for customers

Skills

Backend development

Distributed systems

Java

Go

C/C++

Python

Code reviews

Documentation

Education

Master's Degree

Job description

Introduction

At IBM Software, we transform client challenges into solutions. Building the world’s leading AI-powered, cloud-native products that shape the future of business and society. Our legacy of innovation creates endless opportunities for IBMers to learn, grow, and make an impact on a global scale. Working in Software means joining a team fueled by curiosity and collaboration. You’ll work with diverse technologies, partners, and industries to design, develop, and deliver solutions that power digital transformation. With a culture that values innovation, growth, and continuous learning, IBM Software places you at the heart of IBM’s product and technology landscape. Here, you’ll have the tools and opportunities to advance your career while creating software that changes the world.

Introduction

At IBM Software, we transform client challenges into solutions. Building the world’s leading AI-powered, cloud-native products that shape the future of business and society. Our legacy of innovation creates endless opportunities for IBMers to learn, grow, and make an impact on a global scale. Working in Software means joining a team fueled by curiosity and collaboration. You’ll work with diverse technologies, partners, and industries to design, develop, and deliver solutions that power digital transformation. With a culture that values innovation, growth, and continuous learning, IBM Software places you at the heart of IBM’s product and technology landscape. Here, you’ll have the tools and opportunities to advance your career while creating software that changes the world.

Your Role And Responsibilities

WarpStream is an Apache Kafka compatible data streaming platform built directly on top of object storage with zero local disks. It’s delivered to customers as a BYOC-style offering. The WarpStream team is responsible for this product offering from top to bottom, and as a Senior Engineer you’ll do everything from working on the storage engine running in the Agents, improving the scalability of our multi-tenant control plane, implementing new features like transactions, active-active multi-regions clusters, and even write some HTML/Javascript for our developer console. We’re a small team of < 15 engineers, so you’ll be expected to work across the entire stack from bashing bits at the network level to making sure our UI provides a great developer experience.

You will have an opportunity to solve complex distributed systems problems at scale. You will build services that can operate across different cloud service providers like AWS, GCP, and Azure. You’ll be on-call for all of it.

The WarpStream team is fully remote and geographically distributed, currently we have engineers in: Spain, France, Canada, Ireland, and the USA.

What You Will Do

• Independently drive execution of complex technical projects end to end.
• Build mission-critical backend services that deliver value to our customers. You will play a crucial role in architecting, designing, developing and operationalizing high performance, scalable, reliable and resilient services.
• Troubleshoot and debug technical issues inside a deep and complex technical stack that includes microservices, containers, and virtualization.
• Ensure operational readiness of the services and meet the availability and performance SLA commitments to our customers.

Preferred Education

Master's Degree

Required Technical And Professional Expertise

• 5+ years industry experience designing, building, scaling and supporting backend systems in production with a solid grasp on good software engineering practices such as code reviews, deep focus on quality, and documentation
• Strong programming and algorithmic skills. Proficiency in a major programming language, e.g. Java, Go, C / C++, Python, etc
• Deep curiosity and enthusiasm for distributed systems and storage systems
• Strong focus on project delivery and communication skills
• Experience in driving operational excellence for large production services
• A strong sense of customer centricity, teamwork, technical leadership and mentorship and are excited about team and company success

Preferred Technical And Professional Experience

• Proven track record of delivering large-scale, highly available, high quality systems
• Hands-on technical expertise in large scale systems engineering or distributed systems
• On-call experience handling critical systems