Hadoop SME or Developer

Systechcorp Inc; Pune District, Mumbai; On-site; INR 1,800,000 - 2,200,000; Full time

Job summary

Systechcorp Inc in Mumbai/Pune seeks an experienced Hadoop SME/Developer to drive data migrations and leverage the Hadoop ecosystem. You will design ingestion pipelines for unstructured data and work with HDFS, HBase, Spark, Hive, and Solr while collaborating with customers to translate requirements into scalable solutions.

The ideal candidate has 10+ years IT experience, strong Hadoop expertise, and hands-on programming in Java, Python, Scala, and Bash.

Qualifications

• Total IT experience of 10+ years.
• Minimum 5 years of relevant experience using key Hadoop services.
• Minimum 5 years of relevant experience on HBase.
• Data Integration experience is required.
• Knowledge of Hadoop components: HDFS, HBase, Solr, Spark, Hive.
• Experience in unstructured data processing and validation (documents, images, text).
• Developed data ingestion pipelines in Hadoop for unstructured data.
• Hands-on programming in Java, Python, Scala and Bash scripting.
• Understanding of how structured data is stored in HBase and linked with RDBMS.
• Understanding of Solr text search.
• Experience migrating, storing and retrieving data in object stores like AWS S3 and Azure Blob.
• Excellent communication with customers and prospects.

Responsibilities

• Lead Hadoop migration initiatives and tailor Hadoop solutions.
• Design and implement data ingestion pipelines for unstructured data.
• Collaborate with customers to translate requirements into scalable Hadoop architectures.

Skills

Hadoop

Java

Python

Scala

Bash scripting

Data migrations

Unstructured data processing

Data ingestion pipelines

AWS S3 / Azure Blob

Communication with customers

Tools

HDFS

HBase

Solr

Spark

Hive

Job description

Role: Hadoop SME or Developer

Location: Mumbai/Pune

Experience : 10+ Years

Notice period: Only 15-30days

This position is for a Hadoop SME who will be specializing in Hadoop to data migrations. Good experience of the Hadoop ecosystem will be a key competency for this position.

Minimum Requirements:

• Total IT experience of 10+ years.
• Minimum 5 years of relevant experience using key Hadoop services
• Minimum 5 years of relevant experience on on Hbase
• Data Integration experience is required
• Knowledge of following Hadoop components are required: HDFS, HBase, Solr, Spark, Hive.
• Experience in unstructured data processing and validation such as documents, images and text.
• Worked and developed data ingestion pipelines in Hadoop for unstructured data.
• Good hands-on experience in programming language like Java, Python, Scala and Bash scripting.
• Understanding of how structured data is stored and accessed in HBase and metadata can be linked with the RDBMS.
• Understanding of how Solr text search works.
• Experience in migrating, storing and retrieving data in object stores like AWS S3, and Azure Blob.
• Should be able to communicate effectively with customers and prospects.

Nice to have Experience:

• Working experience with AWS integration services like AWS Glue and Athena.
• Good Experience in Teradata Vantage SQL
• Experience with any database migration would be added plus.