Senior DevOps Engineer – Kafka Specialist

Jobtailor; California (MO); On-site; USD 120,000 - 160,000; Full time

Job summary

Venmo is seeking an experienced data infrastructure engineer to design, implement, and operate secure, scalable data systems for product, analytics, and infrastructure teams.

You will automate provisioning, management, and monitoring of data infrastructure, build scalable pipelines, and implement streaming solutions with AWS SQS and Kafka while collaborating with cross-functional teams on priorities and technical needs.

Qualifications

• 1+ years of relevant experience and a Bachelor's degree or equivalent

Responsibilities

• Design, implement, and operate secure, scalable, reliable, and performant data infrastructure
• Automate provisioning, management, and monitoring of data infrastructure
• Build scalable data pipeline architecture and tools to turn data into insights
• Implement message streaming solutions using AWS SQS and Apache Kafka
• Collaborate with stakeholders on technical issues and infrastructure needs
• Collaborate with teammates on planning and execution of initiatives
• Process and manage relational and NoSQL databases using SQL/NoSQL
• Create large, complex datasets meeting business requirements
• Follow the software development lifecycle conventions
• Contribute to code and design reviews

Skills

Data infrastructure design

AWS SQS and Apache Kafka

Infrastructure as Code

SQL/NoSQL databases

Big data operations

Strong communication

Troubleshooting

Education

Bachelor’s degree in Computer Science or related field

Master’s degree in Computer Science or related field

Tools

AWS

AWS DMS

Luigi

AWS EKS

AWS Glue

Terraform

CloudFormation

Ansible

Python

Java

Bash

AWS CLI

AWS S3

Apache Spark

Job description

• Design, implement, and operate secure, scalable, reliable, and performant data infrastructure for Venmo product, analytics, and infrastructure teams
• Automate provisioning, management, and monitoring of supported data infrastructure
• Build and implement scalable, reliable data pipeline architecture
• Build tools that turn data into actionable insights
• Build and implement message streaming solutions using AWS SQS and Apache Kafka
• Collaborate with Executive, Product, Engineering, and Analytics stakeholders on technical issues and infrastructure needs
• Collaborate with teammates on planning and execution of initiatives
• Process, extract, and manage relational databases and systems using SQL and NoSQL databases
• Create large, complex datasets meeting functional and non-functional business requirements
• Implement tasks within the software development lifecycle
• Follow internal conventions, standards, and procedures
• Contribute to code and design reviews

Requirements

• 1+ years relevant experience and a Bachelor’s degree OR any equivalent combination of education and experience
• Bachelor’s and/or master's in computer science or related field of study
• 3+ years' experience in software development or a related field
• 3+ years' experience operating on-prem/cloud databases/data-warehouses and/or distributed systems 24x7x365
• Experience operating/maintaining message streaming solutions such as AWS SQS and Apache Kafka
• Ability to troubleshoot problems
• Hands-on experience designing and implementing infrastructure for global-scale services
• Hands-on experience with IaaS and PaaS solutions from AWS or similar cloud provider
• Experience with infrastructure as code: AWS CLI, AWS CloudFormation, Terraform, Ansible, Puppet
• Hands-on programming and scripting: Lambda, API GW, Python, Java, Bash
• Hands-on experience with big data operations including AWS S3, AWS DMS, AWS SQS, Luigi, AWS EKS, Apache Spark, AWS Glue
• Strong communication skills and ability to explain technical issues to non-technical audiences
• Understanding of TLS/mTLS protocols is a plus
• AWS Solutions Architect certification is a plus

Core Competencies

Demonstrates expertise in designing and implementing secure, scalable data infrastructure and data pipeline architecture, with hands-on experience in AWS services and message streaming solutions. Proficient in managing relational and NoSQL databases, and skilled in automating infrastructure management using infrastructure as code tools.

Highest-signal resume keywords

• Data Infrastructure Design
• AWS SQS and Apache Kafka
• Infrastructure as Code
• SQL and NoSQL Databases
• Big Data Operations

ATS Optimization Keywords

Hard Skills

• Data Pipeline Architecture
• AWS CLI
• AWS CloudFormation
• Terraform
• Ansible
• Python
• Java
• Bash
• AWS S3
• Apache Spark

Soft Skills

• Strong Communication Skills
• Problem-Solving Ability

Certifications & Qualifications

• AWS Solutions Architect Certification

Industry Keywords

• Distributed Systems
• Cloud Databases
• Message Streaming Solutions
• Data Warehouses
• Software Development Lifecycle

Tools & Technologies

• AWS
• AWS DMS
• Luigi
• AWS EKS
• AWS Glue