Senior Engineer - Data Infrastructure

Engg; San Jose (CA); On-site; USD 140,000 - 190,000; Full time

Job summary

PayPal's Venmo Data Infrastructure team hires a data engineer to advance streaming and data movement across platforms. You will work with Kafka-based architecture and AWS services to build secure, scalable pipelines supporting Venmo product and analytics teams.

Responsibilities include automating provisioning, monitoring, and building tools to derive insights. Strong communication and problem-solving skills are essential for global-scale systems.

Qualifications

• Bachelors and/or master's in computer science or related field.
• 3+ years' experience in software development or a related field.
• 3+ years' experience operating global scale high throughput AWS native data pipelines.

Responsibilities

• Design, implement and operate data infrastructure initiatives to provide secure, scalable, reliable and performant platform to Venmo product, analytics and the other infrastructure teams.
• Automate provisioning, management and monitoring solutions for the supported data infrastructure.
• Build and implement scalable and reliable data pipeline architecture to optimize overall performance.
• Build tools that help users leverage our data into actionable insights to improve our customers' experiences, operational efficiency, and business performance.
• Build and implement message streaming solutions for Venmo applications with AWS SQS, Apache Kafka etc
• Collaborate with stakeholders including the Executive, Product, Engineering and Analytics teams to assist with data-related technical issues and support their data infrastructure need.
• Collaborate with teammates on planning and execution of initiatives.
• Process, extract and manage relational databases and systems such as query authoring (SQL) and NoSQL databases.
• Create large, complex data sets that meet functional and non-functional business requirements.
• Demonstrable ability to learn new technologies and process practices.

Skills

Strong communication

Troubleshooting

Team collaboration

Education

Bachelor's/Master's in CS or related

Tools

Terraform

AWS CloudFormation

Python

Java

Lambda

AWS S3

AWS DMS

AWS EKS

Apache Spark

AWS Glue

AWS SQS

Apache Kafka

Job description

The Company PayPal has been revolutionizing commerce globally for more than 25 years. Creating innovative experiences that make moving money, selling, and shopping simple, personalized, and secure, PayPal empowers consumers and businesses in approximately 200 markets to join and thrive in the global economy. We operate a global, two-sided network at scale that connects hundreds of millions of merchants and consumers. We help merchants and consumers connect, transact, and complete payments, whether they are online or in person. PayPal is more than a connection to third-party payment networks. We provide proprietary payment solutions accepted by merchants that enable the completion of payments on our platform on behalf of our customers. We offer our customers the flexibility to use their accounts to purchase and receive payments for goods and services, as well as the ability to transfer and withdraw funds. We enable consumers to exchange funds more safely with merchants using a variety of funding sources, which may include a bank account, a PayPal or Venmo account balance, PayPal and Venmo branded credit products, a credit card, a debit card, certain cryptocurrencies, or other stored value products such as gift cards, and eligible credit card rewards. Our PayPal, Venmo, and Xoom products also make it safer and simpler for friends and family to transfer funds to each other. We offer merchants an end-to-end payments solution that provides authorization and settlement capabilities, as well as instant access to funds and payouts. We also help merchants connect with their customers, process exchanges and returns, and manage risk. We enable consumers to engage in cross-border shopping and merchants to extend their global reach while reducing the complexity and friction involved in enabling cross-border trade. Our beliefs are the foundation for how we conduct business every day. We live each day guided by our core values of Inclusion, Innovation, Collaboration, and Wellness. Together, our values ensure that we work together as one global team with our customers at the center of everything we do - and they push us to ensure we take care of ourselves, each other, and our communities.

Job Summary:

The Venmo Data Infrastructure team supports Venmo's internal and customer-impacting data movement and message streaming infrastructure. We are actively engaged in partnering with development groups, platform, and infrastructure teams - providing deep subject matter expertise on Kafka-based streaming architecture, and acting as the escalation point for analyzing and resolving complex, large-scale streaming and performance issues.

Essential Responsibilities:

• Implements tasks within the Software Development Lifecycle (SDLC), receiving structure and oversight from more experienced staff
• Follows well-established internal conventions and standard procedures
• Understands internal standards & processes an applies them to make technical decisions
• Collaborates with peers, manager, and project lead to gain understanding of tasks and review solutions
• May contribute to code & design reviews

Expected Qualifications:

• 1+ years relevant experience and a Bachelor's degree OR Any equivalent combination of education and experience.

Additional Responsibilities & Preferred Qualifications :

Data Infrastructure at Venmo The Venmo Data Infrastructure team supports Venmo's internal and customer impacting data movements and message streaming infrastructure. Venmo Data Infrastructure team is actively engaged in partnering with development groups, platform and infrastructure teams, providing deep subject matter expertise to feature design as well as an advocate for analyzing and resolving customer and performance issues at scale.

What You'll Do

• Design, implement and operate data infrastructure's initiatives to provide secure, scalable, reliable and performant platform to Venmo product, analytics and the other infrastructure teams
• Automate provisioning, management and monitoring solutions for the supported data infrastructure
• Build and implement scalable and reliable data pipeline architecture to optimize overall performance.
• Build tools that help users leverage our data into actionable insights to improve our customers' experiences, operational efficiency, and business performance.
• Build and implement message streaming solutions for Venmo applications with AWS SQS, Apache Kafka etc
• Collaborate with stakeholders including the Executive, Product, Engineering and Analytics teams to assist with data-related technical issues and support their data infrastructure need.
• Collaborate with teammates on planning and execution of initiatives.
• Process, extract and manage relational databases and systems such as query authoring (SQL) and NoSQL databases.
• Create large, complex data sets that meet functional and non-functional business requirements.
• Demonstrable ability to learn new technologies and process practices.

What We're Looking For

• Bachelors and/or master's in computer science or related field of study
• 3+ years' experience in software development or a related field
• 3+ years' experience operating global scale high throughput AWS native data pipelines
• Hands-on experience with designing and implementing infrastructure to support global-scale services
• Hands-on experience with IaaS and PaaS solutions from AWS (or similar cloud provider), including: Infrastructure as code (Terraform, AWS CloudFormation) Programming and scripting (Python, Java, Lambda)
• Hands-on experience with related big data infrastructure - AWS S3, AWS DMS, AWS EKS, Apache Spark, AWS Glue - and how they integrate with a streaming backbone
• Experience with operating/maintaining message streaming solutions like AWS SQS, Apache Kafka etc.
• Ability to troubleshoot problems
• Strong communication skills with the ability to understand and explain technical issues to a non-technical audience
• AWS Solutions Architect is a plus

Minimum Qualifications

• 3+ years relevant experien