Kafka Data Integration Engineer #2

HireTalent; New York (NY); On-site; USD 90,000 - 150,000; Full time

Job summary

An established industry player is seeking a skilled Data Engineer with extensive experience in AWS and Apache Kafka. This pivotal role involves designing and operating data solutions, utilizing relational databases, and ensuring efficient data processing. The ideal candidate will have a strong background in AWS services, particularly MSK, and be adept in SQL and Python. Join a dynamic team that values innovation and offers opportunities for growth in a collaborative environment. If you are passionate about data engineering and ready to tackle exciting challenges, this is the perfect opportunity for you.

Qualifications

• Minimum of 7 years of hands-on experience in data engineering.
• Experience with AWS services and Kafka for data processing.

Responsibilities

• Design and operate solutions with relational databases.
• Implement AWS services in distributed computing environments.

Skills

Data Engineering

AWS MSK

Apache Kafka

SQL

Python

Terraform

RDBMS Design

Agile/Scrum

Education

Bachelor’s Degree in Computer Science

Bachelor’s Degree in Software Engineering

Tools

AWS (EC2, ELB, RDS, Route53, S3, VPC, VPN, TGW)

Kafka Connect

Terraform

Job description

Minimum of 7 years of hands-on experience within data engineering organization;
• Experience designing and operating solutions with relational Databases (Oracle, Postgres, SQL Server);
• Self-motivated and able to work with minimal supervision;
• Familiar with the creation of different types of documentation like functional, process, test plan etc;
• Experience in AWS MSK (Amazon Managed Streaming for Apache Kafka);
• Well-versed on MSK Connect and Kafka Connect for integration with RDBMS, using JDBC connectors;
• Hands-on experience in Terraform;
• Experience in primary AWS services (EC2, ELB, RDS, Route53, S3, VPC, VPN, TGW);
• Strong AWS skills;
• Strong track record of implementing AWS services in a variety of distributed computing environments;
• Ability to install, maintain and troubleshoot Kafka;
• Experience working with Apache Kafka, like designing, developing, and managing data streams, troubleshooting, internals, cluster design, optimization, monitoring, schema registry and schema evolution, REST API, and its many related components;
• Extensive experience with messaging and stream processing on Kafka;
• In-depth knowledge of all the functionalities surrounding Kafka;
• Excellent understanding of SQL and Python for data processing;
• Understanding of Data Modeling concepts and RDBMS Database design;
• Ability to work with a variety of platforms and application stacks;
• Experience working in an Agile/Scrum development process;
• Bachelor’s Degree in Computer Science or Software Engineering.


All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender, identity, national origin, disability, or protected veteran status.