Senior Software Engineer, Data Platforms

eBay; United States; Remote; USD 90,000 - 150,000; Full time

Job summary

Join a forward-thinking company as a Senior Software Engineer in Data Platforms, where you will lead the design of innovative data architectures that support real-time and batch processing. Collaborate with a team of passionate engineers and data scientists to create cutting-edge data solutions. This role offers the opportunity to work with advanced streaming technologies like Apache Kafka and Flink, while optimizing data consumption and pipeline automation. If you're ready to make a significant impact in the world of e-commerce and data engineering, this is the perfect opportunity for you.

Qualifications

• 5+ years of experience in data engineering with a focus on streaming technologies.
• Strong understanding of data semantics and automated data pipelines.

Responsibilities

• Design and enhance data architecture for real-time and batch processing.
• Implement systems for effective data semantics management.
• Develop automated data pipelines for efficient data flow.

Skills

Apache Kafka

Apache Flink

Lakehouse Architecture

Data Semantics

Data Discovery

Pipeline Automation

Java

Scala

Python

Job description

Senior Software Engineer, Data Platforms

At eBay, we're more than a global ecommerce leader — we’re changing the way the world shops and sells. Our platform empowers millions of buyers and sellers in more than 190 markets around the world. We’re committed to pushing boundaries and leaving our mark as we reinvent the future of ecommerce for enthusiasts.

Our customers are our compass, authenticity thrives, bold ideas are welcome, and everyone can bring their unique selves to work — every day. We're in this together, sustaining the future of our customers, our company, and our planet.

Join a team of passionate thinkers, innovators, and dreamers — and help us connect people and build communities to create economic opportunity for all.

About the team and the role:

Cloud Data Technologies (CDT) team at eBay is responsible for all the data infrastructure and management of E2E data lifecycle for all eBay data from onsite to analytics use cases. We are looking for a senior software engineer deeply passionate about data-driven technologies and architectures.

The ideal candidate will have a robust background in streaming technologies such as Apache Kafka and Apache Flink, coupled with extensive experience in lakehouse architecture. This role demands a strong understanding of data semantics, data discovery, pipeline automation, materialized views, and subsumption within large-scale data environments.

What you will accomplish:

• Architectural Leadership: Design and enhance our data architecture to support real-time and batch processing with a focus on scalability and fault tolerance using Kafka, Flink, and lakehouse principles.
• Data Semantics and Discovery: Implement systems and procedures for effective data semantics management, ensuring data is accurately categorized and easily discoverable.
• Pipeline Automation: Develop and maintain automated data pipelines that ensure efficient data flow and processing from multiple sources to our lakehouse architecture.
• Data Consumption Optimization: Create strategies and systems for optimal generation of materialized views and data subsumption to ensure that our data architecture remains cutting-edge, minimizes redundancy, and achieves required level of performance.
• Cross-functional Collaboration: Work closely with data scientists, business analysts, and other engineering teams to define and refine requirements that drive data solutions.
• Innovation and Research: Stay abreast of the latest industry developments in data engineering and propose adoption of new technologies or methodologies to keep our data infrastructure ahead of the curve.

What you will bring:

• Extensive experience with Apache Kafka, Apache Flink, and other relevant streaming technologies.
• Deep understanding of lakehouse architecture and its implementation in large-scale environments.
• Strong knowledge of data semantics, discovery processes, and data governance.
• Proven ability to design and implement automated data pipelines and materialized views.
• Experience with data subsumption techniques and tools.
• Proficiency in Java, Scala, Python, or similar languages.
• 5+ years of directly applicable experience

#LI-AZ1

Please see the Talent Privacy Notice for information regarding how eBay handles your personal data collected when you use the eBay Careers website or apply for a job with eBay.

eBay is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, sex, sexual orientation, gender identity, veteran status, and disability, or other legally protected status. If you have a need that requires accommodation, please contact us at [e-mail]. We will make every effort to respond to your request for accommodation as soon as possible. View our accessibility statement to learn more about eBay's commitment to ensuring digital accessibility for people with disabilities.

About the company

eBay Inc. is an American multinational e-commerce corporation based in San Jose, California, that facilitates consumer-to-consumer and business-to-consumer sales through its website.