Hadoop Admin on w2 contract

Deegit Inc; Chicago (IL); On-site; USD 100,000 - 130,000; Full time

Job summary

A technology solutions company is looking for a skilled Hadoop Admin in Chicago, IL. The role involves managing Hadoop infrastructure, optimizing big data solutions, and ensuring comprehensive security for data processes. Candidates should have strong experience with Hadoop flavors like Cloudera and manage cloud platforms such as Azure or AWS. Additional knowledge in data ingestion and tools like Tableau and PowerBI is beneficial. This position offers the opportunity to work on innovative big data projects in a dynamic environment.

Qualifications

• Experience with at least one popular Hadoop flavor.
• Solid experience managing & administering cloud platforms.
• Knowledge of data ingestion processes.

Responsibilities

• Manage Hadoop infrastructure and performance optimization.
• Establish big data security protocols.
• Configure cloud and Big Data solutions.

Skills

Knowledgeable in SQL server DB

Aggressive self-starter

IOT experience

Experience with Hadoop flavors

Solid experience managing cloud platforms

Big data security expertise

Data ingestion experience

Familiarity with data visualization tools

Scheduler and runtime environment configuration

Big data modeling experience

Tools

Azure

AWS

Hadoop

Cloudera

HortonWorks

MapR

Neo4J

Tableau

PowerBI

R

SAS

Job description

Job Description

Hadoop Admin

Long term

Chicago, IL

Mandatory Skills:

• Knowledgeable in SQL server DB
• Aggressive self-starter, can operate without Infrastructure support
• IOT experience

Desired Skills:

• 1. a. Config DB backup and recovery processes
• b. File system (Blob & HDFS) management and monitoring
• c. Manage and review Hadoop log files; house keeping
• d. Experience with at least one popular Hadoop flavor – Cloudera, HortonWorks, MapR
• e. Installation and Config of Hadoop projects such as Hive, Pig, HBase, Spark, etc.
• f. Performance optimization (MR job and other)
• g. Setup performance monitoring & tuning

Cloud

• 1. a. Solid experience managing & administering cloud platform - Azure, AWS, BigTable, etc.
• b. deploying Big Data solutions in Cloud
• c. Cluster install & config
• d. Capacity estimation and configure for elasticity
• e. Script automation for allocation, deallocation, etc.

Establish Big Data Security

• 1. a. Data at Rest & Transit
• b. Access from backend and front-end applications
• c. Row level security & Role based security
• d. integration - LDAP, AD, Okta

Ingestion

• 1. a. Sqoop, Kakfa, Storm – highly desirable
• b. ADF, Informatica Cloud, etc. are definite plus
• c. Setup processes for historical and incremental data ingestion

Front-end access – Configure connectivity and interfaces enabling Cognos, Tableau, PowerBI, R, and SAS applications to access Hadoop Big data.

Scheduler & Runtime environment – Configure YARN and Oozie for optimal resource utilization and end-to-end workflow process.

Deploy and configure Neo4J (Graph DB), Document DB and emerging big data repositories

Big data modeling

Qualifications

Experience with at least one popular Hadoop flavor – Cloudera, HortonWorks, MapR

Installation and Configration of Hadoop projects