Senior Data Engineer - Real-Time Data Pipelines & Big Data

TechDigital Group; Atlanta (GA); On-site; USD 100,000 - 130,000; Full time

Job summary

A leading technology company in Atlanta is seeking a Senior Data Engineer who will design and develop cutting-edge Big Data solutions. This role requires hands-on experience with technologies like Hadoop and Spark, along with programming expertise in Java, Scala, or Python. The ideal candidate will work collaboratively to optimize data pipelines and contribute to the company's innovative data strategies.

Qualifications

• Hands-on experience with Big Data and analytics technologies.
• Programming expertise in Java, Scala, or Python.
• Real-time and batch data processing skills.
• Proficiency in data modeling and schema design.
• Exposure to cloud technologies.

Responsibilities

• Design & develop batch and streaming solutions.
• Assess technologies for ingestion and storage.
• Develop software utilizing open-source technologies.
• Identify functional and system requirements.

Skills

Hadoop

Spark

Java

Scala

Python

Data modeling

ETL workflows

Agile/Scrum

Education

BS/BA degree in Computer Science

Tools

Databricks

Cloudera

Job description

A leading technology company in Atlanta is seeking a Senior Data Engineer who will design and develop cutting-edge Big Data solutions. This role requires hands-on experience with technologies like Hadoop and Spark, along with programming expertise in Java, Scala, or Python. The ideal candidate will work collaboratively to optimize data pipelines and contribute to the company's innovative data strategies.