Senior Big Data & Spark Developer

SmartRecruiters, Inc.; Atlanta (GA); In loco; USD 66.000 - 88.000; Tempo pieno

Descrizione del lavoro

Global Channel Management is seeking a Hadoop Developer with 5+ years of Python and Java/Scala experience to join our data team in Atlanta. The role requires strong expertise in SPARK, big data projects, and data modeling within a enterprise environment.

You will translate complex requirements into detailed design, develop SPARK-based solutions, and load disparate data sets using Kafka and related ETL tools. Experience with Cloudera/Hortonworks is valued, with exposure to Teradata a plus.

Competenze

• Hadoop Developer needs 5+ years of Python, Java/ Scala.
• B.S. and M.S. in mathematics, computer science or engineering.
• 3+ years of demonstrated technical proficiency with SPARK, big data projects, and experience in data modeling.
• Experience designing and developing data ingestion and processing/transformation frameworks leveraging tools such as ADF, NiFi, Sqoop, Eclipse.
• Should have worked in Big data space for at least 5 years in Apache Spark, Hive and MapReduce.
• Experience with a variety of Big Data ETL tools, e.g. Spark, Hive, Kafka, Sqoop, MapReduce, Scala, Zookeeper, Etc
• Should have experience in Cloudera or Hortonworks and exposure to Teradata is beneficial.

Mansioni

• Translate complex functional and technical requirements into detailed design
• Proficiency in SPARK for technical development and implementation
• Load disparate data sets by leveraging various big data technology (Kafka)

Conoscenze

Python

Java/Scala

SPARK

Big data

Formazione

B.S. and M.S. in mathematics, computer science or engineering

Strumenti

ADF

NiFi

Sqoop

Eclipse

Kafka

Zookeeper

Descrizione del lavoro

Global Channel Management is seeking a Hadoop Developer with 5+ years of Python and Java/Scala experience to join our data team in Atlanta. The role requires strong expertise in SPARK, big data projects, and data modeling within a enterprise environment.

You will translate complex requirements into detailed design, develop SPARK-based solutions, and load disparate data sets using Kafka and related ETL tools. Experience with Cloudera/Hortonworks is valued, with exposure to Teradata a plus.