Big Data Consultant

Job Responsibilities

1. Must have good technical experience and should be able to understand and develop the modules and bring the task to closure on time.
2. At least 4+ years of development work experience in Hadoop programming (HDFS) using PySpark/Scala/Java with Hive-based Data warehouse projects, along with good Shell Scripting experience.
3. Able to understand the requirements and ingest the data into the Big Data platform using Spark with HIVE.
4. Should be able to write SQL scripts, including complex SQL scripts to process the data with SCD handling exposure.
5. Responsible for meeting quality and scope as per the project plans, must be able to meet stringent timelines.
6. Flexible to work as per regional timing.
7. Excellent communication and documentation skills.
8. Very good at team playing and flexible to work with the offshore team in different time zones based on project needs.
9. Willingness to obtain certification in Databricks Certified Developer: Apache Spark 3.X.

Nice to Have

1. Proactive, with good communication skills to articulate technical issues.
2. Exposure to Confluence/JIRA.
3. Ability to work independently; prior experience with databases like Oracle/SQL Server/ETL will be an added advantage.
4. At least 1+ years of Module Lead experience.
5. Hands-on experience in data analysis and debugging SQL issues.
6. Experience in performance tuning Hive QLs and cluster nodes, as well as functional testing, test planning, and execution.