Data Engineer

ALLTECH CONSULTING SVC INC; Alpharetta (GA); On-site; USD 90,000 - 120,000; Full time

Job summary

A technology consulting firm in Georgia is seeking a Streaming Data Engineer to design and develop ETL pipelines and work with large-scale data systems. The ideal candidate has at least 5 years of application development experience, with a focus on data engineering using Kafka, and strong coding skills in Python or Java. This role involves working on distributed systems architecture, and the ability to learn new technologies is essential.

Qualifications

• 5 years of application development experience required.
• At least 2 years of data engineering experience with Kafka.
• Experience writing applications on Linux.

Responsibilities

• Understand distributed systems architecture.
• Design and develop ETL pipelines.
• Lead full cycle of development including requirements, design, and implementation.

Skills

Application development experience

Data engineering with Kafka

Coding in Python

Coding in Java

SQL and database experience

Tools

Linux

AWS

ElasticSearch ELK

Job description

Job Description:

• Real‑time operation intelligence team in our Company Enterprise Computing is responsible for streaming terabytes of data daily.
• We have built job frameworks to run large‑scale ETL pipelines with Kafka, ElasticSearch ELK, Snowflake, Hadoop.
• Our applications run both on‑prem and on the cloud. There are hundreds of dashboards built for business and operations to provide insight and actionable items in real time.

We are looking for streaming data engineer

• Understand distributed systems architecture, design.
• Design and develop ETL pipelines with a wide range of technologies.
• Able to work on full cycle of development including defining requirements, design, implementation, testing, deployment.
• Strong communication skills to collaborate with various teams.
• Able to learn new technologies and work independently.

Requirements:

• 5 years of application development experience, at least 2 years data engineering with Kafka.
• Working experience writing and running applications on Linux.
• 5 years of coding experience with at least one of the languages: Python, Ruby, Java, C/C++, Go.
• SQL and database experience.

Optional:

• AWS or other cloud technologies.
• ElasticSearch ELK.