Big Data Consultant

HR Tech; Singapore; On-site; SGD 70,000 - 100,000; Full time

Job summary

A leading tech company in Singapore is seeking an experienced developer with strong Hadoop programming skills. The role requires at least 4 years of experience in Hadoop using PySpark/Scala/Java, along with good skills in SQL and Shell Scripting. The candidate should have excellent communication skills and be able to deliver projects on time, working flexibly with offshore teams across different time zones.

Qualifications

• At least 4+ years of development experience in Hadoop programming.
• Good SQL skills for complex data processing.
• Ability to work across different time zones.

Responsibilities

• Develop modules and ensure timely delivery.
• Ingest data into the Big Data platform using Spark with HIVE.
• Write complex SQL scripts and handle data processing.

Skills

Hadoop programming (HDFS)

PySpark/Scala/Java

Shell Scripting

SQL scripts

Excellent communication skills

Team player

Tools

Confluence

JIRA

Oracle/SQL Server

Job description

• Must have good technical experience and should be able to understand and develop the modules and bring the task to closure on time.
• At least 4+ years of development work experience in Hadoop programming (HDFS) using PySpark/Scala/Java with Hive-based Data warehouse projects. And, with good Shell Scripting experience.
• Able to Understand the requirements and ingest the data into the Bigdata platform using Spark with HIVE.
• Should be able to write the SQL scripts and write few complex SQL Scripts to process the data with SCD handling exposure.
• Responsible for meeting the quality, and scope as per the project plans must be able to meet the stringent timelines.
• Flexible to work as per the regional timing.
• Excellent communication and documentation skills
• Very good at team playing and flexibility to work with the Offshore team in different time zones based on project needs.
• Willingness to do Certification in Databricks Certified Developer: Apache Spark 3. X

Nice to Have

• Proactive, and have good communication skills to articulate technical issues.
• Exposure to is Confluence/JIRA
• Should be able to work independently Prior Experience to any Databases like Oracle/SQL Server/ETL will be an added advantage.
• At least 1+ years of Module Lead Experience
• Hands-on experience in data analysis and debugging the issues in SQL.
• Performance tuning the Hive QLs and the Cluster nodes Functional Testing Functional Test plan and execution experience.