Principal MTS, Software Engineer

eBay; Bengaluru; On-site; INR 6,000,000 - 9,000,000; Full time

Job summary

eBay is seeking an experienced Software Engineer/Architect with a strong Hadoop background to design and develop its Hadoop infrastructure. You will oversee distributed storage (HDFS), compute (Spark), scheduling (Yarn/YuniKorn), and data formats (Iceberg/Hudi) to ensure reliable, scalable, high-performance systems powering mission-critical applications.

Responsibilities include leading architecture enhancements for scalability and availability, driving end-to-end data processing pipelines, and

Qualifications

• 15+ years of software design and distributed systems experience.
• Expert-level proficiency in Java and/or Scala.
• Experience with Hadoop, Spark, Iceberg, and YuniKorn.

Responsibilities

• Architect Distributed Scalability & High Availability across Hadoop/open-source platforms to support petabyte-scale growth.
• Own end-to-end technical execution of data pipelines and platform frameworks for reliability and throughput.
• Design and deliver internal and customer-facing tooling for data access, governance, and cluster management.
• Promote interoperability between Hadoop data layers and broader enterprise platforms to automate workflows.

Skills

Java/Scala

Distributed Systems

Performance Tuning

Root Cause Analysis

Linux/Unix

Tools

Hadoop

Spark

Iceberg

YuniKorn

Job description

At eBay, we're more than a global ecommerce leader - we're changing the way the world shops and sells. Our platform empowers millions of buyers and sellers in more than 190 markets around the world. We're committed to pushing boundaries and leaving our mark as we reinvent the future of ecommerce for enthusiasts.

Our customers are our compass, authenticity thrives, bold ideas are welcome, and everyone can bring their unique selves to work - every day. We're in this together, sustaining the future of our customers, our company, and our planet.

Join a team of passionate thinkers, innovators, and dreamers - and help us connect people and build communities to create economic opportunity for all.

We are seeking an experienced Software Engineer/Architect with a strong background in the Hadoop related ecosystem/distributed system, responsible for the design and development of the Hadoop infrastructure. This role also involves development and operations of eBay's Hadoop infrastructure including the distributed storage(HDFS), distributed compute(Spark), distributed scheduling(Yarn/Yunikorn) and advanced data format(Iceberg/Hudi/etc) to ensure our distributed systems are reliable, scalable, and high performing to serve as the backbone for eBay's mission-critical applications.

Responsibilities

• Architect Distributed Scalability & High Availability: Lead architectural enhancements across massive Hadoop and open-source data platforms, ensuring elite system availability, fault tolerance, and zero-downtime scaling to support petabyte-scale growth and strategic enterprise initiatives.
• Drive High-Impact Infrastructure Initiatives: Own end-to-end technical execution of complex data processing pipelines and platform frameworks, optimizing resource utilization, throughput, and performance reliability across distributed environments.
• Engineer Developer & Operational Tooling: Design and deliver advanced internal and customer-facing tools that streamline data access, governance, and cluster management, empowering cross-functional engineering teams to extract insights efficiently.
• Architect Ecosystem Integration: Champion seamless interoperability between Hadoop data layers and broader enterprise platforms, eliminating operational silos and automating workflows to foster a cohesive, data-driven organizational ecosystem.

Qualifications & Technical Expertise

• Extensive Engineering Experience: 15+ years of software design and distributed systems development, backed by a deep mastery of open-source technologies, large-scale cluster architectures, and enterprise production environments.
• Advanced Programming & Computer Science Foundations: Expert-level proficiency in Java and/or Scala, with a robust command of algorithms, concurrency models, and data structures tailored for high-performance, low-latency distributed computing.
• Distributed Systems & Infrastructure Depth: Comprehensive mastery of Linux/Unix internals, network topologies, and distributed computing principles required to diagnose and resolve complex multi-node failure modes and bottlenecks.
• Big Data Ecosystem Mastery: Hands-on architectural expertise with modern big data frameworks (Hadoop, Spark, Apache Iceberg, Apache YuniKorn); active history of open-source project contributions or upstream community engagement is strongly preferred.
• Complex Troubleshooting & Performance Tuning: Proven diagnostic acumen in root-cause analysis, memory profiling, and systemic performance tuning for large-scale distributed defects and throughput constraints.
• Delivery & Execution Leadership: Results-driven technologist with a history of successfully shipping complex infrastructure products in fast-paced, high-velocity enterprise environments without compromising quality.

eBay is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, sex, sexual orientation, gender identity, veteran status, and disability, or other legally protected status. If you have a need that requires accommodation, please contact us at [e-mail]. We will make every effort to respond to your request for accommodation as soon as possible. View our accessibility statement to learn more about eBay's commitment to ensuring digital accessibility for people with disabilities.