Senior Big Data Engineer (Hadoop/AWS) - Remote in 6 months

Mapjects.com; San Francisco (CA); Hybrid; USD 120,000 - 160,000; Full time

Job summary

SmartRecruiters, Inc. is seeking a senior software engineer to lead Hadoop-based data processing initiatives in the DC/VA region. You will leverage Java and Hadoop to build scalable map/reduce jobs and contribute to the broader data platform.

The role emphasizes designing, implementing, and extending the Hadoop ecosystem with Hive/HBase, while collaborating with Site-Operations on cluster upgrades. A remote option may be available after an initial period.

Qualifications

• Experience designing and implementing Hadoop-based solutions.
• Experience with HDFS/MapReduce/Hive/HBase is preferred.

Responsibilities

• Design and implement map reduce jobs to support distributed processing using Java, Hive and HBase.
• Build libraries, user defined functions, and frameworks around Hadoop.
• Research, evaluate and utilize new technologies/tools around Hadoop ecosystem such as MapReduce, HDFS, Hive, HBase, Flume, Mapjects Engine etc.
• Develop user defined functions to provide custom HBase/Hive capabilities.
• Work with Site-Operations team on upgrades of the cluster.
• Participate in the installation, configuration and administration of single-node and multi-node Hadoop HDFS cluster.

Skills

Hadoop

MapReduce

Java

HDFS

Hive

HBase

Job description

SmartRecruiters, Inc. is seeking a senior software engineer to lead Hadoop-based data processing initiatives in the DC/VA region. You will leverage Java and Hadoop to build scalable map/reduce jobs and contribute to the broader data platform.

The role emphasizes designing, implementing, and extending the Hadoop ecosystem with Hive/HBase, while collaborating with Site-Operations on cluster upgrades. A remote option may be available after an initial period.