Senior DevOps Engineer: Kafka & Data Pipeline Architect

Jobtailor; California (MO); On-site; USD 120,000 - 160,000; Full time

Job summary

Venmo is seeking an experienced data infrastructure engineer to design, implement, and operate secure, scalable data systems for product, analytics, and infrastructure teams.

You will automate provisioning, management, and monitoring of data infrastructure, build scalable pipelines, and implement streaming solutions with AWS SQS and Kafka while collaborating with cross-functional teams on priorities and technical needs.

Qualifications

• 1+ years of relevant experience and a Bachelor's degree or equivalent

Responsibilities

• Design, implement, and operate secure, scalable, reliable, and performant data infrastructure
• Automate provisioning, management, and monitoring of data infrastructure
• Build scalable data pipeline architecture and tools to turn data into insights
• Implement message streaming solutions using AWS SQS and Apache Kafka
• Collaborate with stakeholders on technical issues and infrastructure needs
• Collaborate with teammates on planning and execution of initiatives
• Process and manage relational and NoSQL databases using SQL/NoSQL
• Create large, complex datasets meeting business requirements
• Follow the software development lifecycle conventions
• Contribute to code and design reviews

Skills

Data infrastructure design

AWS SQS and Apache Kafka

Infrastructure as Code

SQL/NoSQL databases

Big data operations

Strong communication

Troubleshooting

Education

Bachelor’s degree in Computer Science or related field

Master’s degree in Computer Science or related field

Tools

AWS

AWS DMS

Luigi

AWS EKS

AWS Glue

Terraform

CloudFormation

Ansible

Python

Java

Bash

AWS CLI

AWS S3

Apache Spark

Job description

Venmo is seeking an experienced data infrastructure engineer to design, implement, and operate secure, scalable data systems for product, analytics, and infrastructure teams.

You will automate provisioning, management, and monitoring of data infrastructure, build scalable pipelines, and implement streaming solutions with AWS SQS and Kafka while collaborating with cross-functional teams on priorities and technical needs.