Staff Streaming/Messaging Infra Engineer

eBay; Bengaluru; On-site; INR 2,500,000 - 3,600,000; Full time

Job summary

eBay is seeking an infrastructure engineer to own the control and data planes for real-time streaming platforms, including Apache Flink and Apache Kafka, across multi-cluster Kubernetes environments. You will drive scalable lakehouse storage design, ensure high availability, and enhance observability while collaborating with global teams.

The role emphasizes hands-on development of production-grade software with AI coding tools and a strong foundation in distributed systems, networking, and OS

Qualifications

• Strong CS fundamentals: data structures, concurrency, OS internals, networking.
• Hands-on experience with Apache Flink at scale (state backend, checkpointing, custom operators).
• Experience with Apache Kafka (or Pulsar/RocketMQ) including broker architecture and partitioning.
• Experience running high-throughput stateful workloads on Kubernetes across hybrid/private clouds.
• Familiarity with streaming table formats and storage engines (e.g., Apache Iceberg, Paimon).

Responsibilities

• Design, build, and operate core control and data planes for Flink and Kafka on multi-cluster Kubernetes across DCs/AZs.
• Lead the development of real-time streaming lakehouse storage architecture for low-latency analytics at scale.
• Ensure high availability, fault recovery, and zero-downtime operations for large-scale streaming topologies.
• Advance observability with monitoring, tracing, alerts, and failure diagnosis tooling.
• Collaborate with global teams to shape the future of eBay's real-time data ecosystem.

Skills

Java

Rust

C++

Data structures

Concurrency

Networking

Apache Flink

Apache Kafka

Multi-cluster Kubernetes

Education

Bachelor's degree in CS or related field

Tools

Kubernetes

Docker

Job description

At eBay, we're more than a global ecommerce leader - we're changing the way the world shops and sells. Our platform empowers millions of buyers and sellers in more than 190 markets around the world. We're committed to pushing boundaries and leaving our mark as we reinvent the future of ecommerce for enthusiasts.

Our customers are our compass, authenticity thrives, bold ideas are welcome, and everyone can bring their unique selves to work - every day. We're in this together, sustaining the future of our customers, our company, and our planet.

Join a team of passionate thinkers, innovators, and dreamers - and help us connect people and build communities to create economic opportunity for all.

About the Team and the Role

Join the infrastructure engineering team responsible for building, scaling, and operating eBay's core messaging and real-time streaming platforms. Our infrastructure powers mission-critical, high-throughput applications across eBay's global ecosystem.

In this role, you will act as a core developer and architect, taking full ownership of both the control plane and data plane for our distributed streaming platforms—including Apache Flink and Apache Kafka—deployed across multiple private cloud Kubernetes clusters spanning various Availability Zones (AZs) and Data Centers (DCs). You will also play a key role in designing and evolving our next-generation streaming lakehouse storage systems.

What You Will Accomplish

• Architect & Scale Infrastructure: Design, build, and operate eBay's core control plane and data plane for Apache Flink and Apache Kafka on multi-cluster Kubernetes environments across multiple DCs/AZs.
• Build Streaming Lakehouses: Lead the design and development of our real-time streaming lakehouse storage architecture, enabling low-latency analytical workloads at massive scale.
• Ensure Operational Excellence: Drive high availability, automated fault recovery, and zero-downtime operations for large-scale production streaming topologies.
• Enhance System Observability: Advance enterprise monitoring, distributed tracing, automated alerting, and failure-diagnosis tooling to maximize platform uptime.
• Collaborate Globally: Partner with cross-functional architecture and platform teams worldwide to shape the future of eBay's real-time data ecosystem.
• Demonstrated proficiency building production-grade software with AI coding agents, improving engineering velocity while maintaining high standards for quality, availability, and maintainability.
• Ability to decompose problems and provide agents with the right context, specifications, constraints, and validation criteria.
• Ability to capture and express business, architecture, codebase, and operational knowledge so AI coding agents can execute complex tasks accurately and autonomously.
• Hands-on experience delivering complex, production-quality software using AI coding tools such as Claude Code, Codex, etc.

What You Will Bring

• Core Distributed Systems Experience: Deep hands-on background in computer science fundamentals (data structures, concurrency, OS internals, and networking) with strong expertise in Java, Rust, or C++.
• Stream Processing Engine Expertise: Strong hands-on experience developing, tuning, and operating Apache Flink at scale (e.g., state backend management, checkpointing tuning, custom operators, Flink Kubernetes Operator).
• Distributed Messaging Deep-Dive: Proven experience with Apache Kafka (or Pulsar/RocketMQ), including core broker architecture, partition strategy, and control/data plane management.
• Container & Multi-Cluster Orchestration: Solid experience running high-throughput stateful workloads on Kubernetes across hybrid or private cloud environments.
• Streaming Lakehouse Knowledge: Familiarity with modern streaming table formats and storage engines such as Apache Paimon, Apache Iceberg, or Apache Fluss.

Nice to Have

• Direct experience writing custom Kubernetes operators or control planes for distributed data systems.
• Deep knowledge of embedded storage engines (RocksDB, LevelDB) and memory management.
• Active contributions to open-source projects in the streaming ecosystem (e.g., Apache Flink, Apache Kafka, Apache Paimon).

Additional Details

eBay is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, sex, sexual orientation, gender identity, veteran status, and disability, or other legally protected status. If you have a need that requires accommodation, please contact us at [e-mail]. We will make every effort to respond to your request for accommodation as soon as possible. View our accessibility statement to learn more about eBay's commitment to ensuring digital accessibility for people with disabilities.

We use cookies to enhance your experience and may use AI tools for administrative tasks in the hiring process. To learn how we handle your personal data and use AI responsibly, please visit our Talent Privacy Notice, Privacy Center, and AI Hiring Guidelines.