Hadoop Developer

BinTech Group; Atlanta (GA); Hybrid; USD 110,000 - 160,000; Full time

Job summary

BinTech Group seeks a Data Engineer to support the Data and Analytics Platform, information management, and solution delivery. You will ensure consistent design across data flows and systems, build data transformation processes, metadata handling, data quality controls, and workload management.

You may mentor junior engineers and collaborate with Architects, Developers, Analysts, QA, and stakeholders to drive delivery and release activities.

Qualifications

• 3–6 years' experience with Hadoop stack and storage technologies
• Extensive knowledge on Bigdata Enterprise architecture (Cloudera preferred)
• Experience with NoSQL technologies (Cassandra/MongoDB) and SQL/ETL
• Strong analytical skills and interest in algorithms; capable of team delivery
• Experience with distributed computing principles and data analysis across multiple systems
• Experience with scripting (Unix/Linux) and scheduling (Autosys)
• Ability to mentor junior engineers and collaborate with architects, developers and analysts

Responsibilities

• Support Data and Analytics Platform, Information Management and Solution Delivery
• Ensure design/engineering approach for data solutions is consistent across flows and systems
• Define data management controls, identify gaps, and implement plans to close them
• Lead experimentation and testing to validate solutions and document deployment requirements

Skills

Analytical mindset

Team player

Self-starter

Communication skills

Education

Degree in Computer Science or equivalent

Tools

Hadoop stack

HDFS

MapReduce

Yarn

HIVE

Sqoop

Impala

Spark

Flume

Kafka

Oozie

HBase

RDBMS

SQL

NoSQL (Cassandra/MongoDB)

Unix/Linux scripting

Autosys

Python (Django/Flask)

Tableau

Kerberos

Job description

Responsibilities

• Support Data and Analytics Platform, Information Management and Solution Delivery
• Ensures design and engineering approach for complex data solutions is consistent across multiple flows and systems, while building processes to support data transformation, data structures, metadata, data quality controls, dependency and workload management
• Responsible to define internal controls, identify gaps in data management standards adherence and work with appropriate partners to develop plans to close the same, lead concept and experimentation testing to synthesize the results and validate and improve the solution, document and communicate required information for deployment, maintenance, support, and business functionality
• May be required to mentor more junior Data Engineers and coach team members in delivery/release activities

Requirements

• 3-6 years' experience in Hadoop stack and storage technologies, HDFS, MapReduce, Yarn, HIVE, sqoop, Impala , spark, flume, kafka and oozie
• Extensive Knowledge on Bigdata Enterprise architecture (Cloudera preferred)
• Excellent analytical capabilities - Strong interest in algorithms
• Experienced in HBase, RDBMS, SQL, ETL and data analysis
• Experience in No SQL Technologies (ex., Cassandra/ MongoDB, etc.)
• Experienced in scripting(Unix/Linux) and scheduling (Autosys)
• Experience with team delivery/release processes and cadence pertaining to code deployment and release
• Research oriented, motivated, pro-active, self-starter with strong technical, analytical and interpersonal skills
• A team player with good verbal and written skills, capable of working with a team of Architects, Developers, Business/Data Analysts, QA and client stakeholders
• Versatile resource with balanced development skills and business acumen to operate at a fast and accurate speed
• Proficient understanding of distributed computing principles. Continuously evaluate new technologies, innovate and deliver solution for business-critical applications

HDFS, MapReduce, Yarn, HIVE, sqoop, Impala , spark, flume, kafka and oozie, HBase, RDBMS, SQL, ETL and data analysis

Desired skills

• Object-oriented programming and design experience
• Degree in Computer Science or equivalent
• Experience with automated testing methodologies and frameworks, including JUnit
• Python IDEs (Django, Flask), data wrangling and analytics in a python based environment
• Fundamentals of Python Data Structures, Collections, Pandas for file and other type of data handling, visualizations, etc.
• Visual Analytics Tools knowledge (Tableau)
• Experience with Big Data Analytics & Business Intelligence and Industry standard tools integrated with Hadoop ecosystem (R , Python)
• Data Integration, Data Security on Hadoop ecosystem (Kerberos)
• Any Big Data certification (ex., Cloudera's CCP, CCA)