Big Data Developer

Tech Aalto Pte ltd; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

Tech Aalto Pte ltd is seeking a Big Data Developer to design and implement scalable data processing solutions within the Hadoop ecosystem in Singapore. The role requires hands-on experience with Hive, Spark, Scala, PySpark, and data warehousing concepts.

Responsibilities include developing with Hadoop components, building automated deployments, and using tools like Jenkins, Control-M, Grafana, Bitbucket, Jira and ADO DevOps.

Qualifications

• Good understanding of concurrent software systems and building them in a way that is scalable, maintainable, and robust.

Skills

Big Data

Data warehousing

Shell scripting

Python

Scala

REST services

Tools

Hive

HDFS

Spark

Spark SQL

Parquet

ORC

Jenkins

Control-M

Grafana

Bitbucket

Jira

ADO DevOps

Maven

Dataiku

Unix Shell

Python

Tez

MR

Oracle

Job description

Big Data Developer

Experience: (3 – 10 Years)

Job Requirements:

• Good understanding of concurrent software systems and building them in a way that is scalable, maintainable, and robust
• Experience in designing application solutions in hadoop ecosystem
• Deep understanding of the concepts in Hive, HDFS, yarn, Spark, Spark sql, Scala and Pyspark
• HDFS file formats and their use cases (eg Parquet, ORC, Sequence etc)
• Good knowledge in data warehousing system
• Experienced in any scripting language (SHELL, PYTHON)
• HortonWorks distribution and understanding on SQL engines (Tez, MR)
• Java/RestServices/Maven experience is a value addition
• Control-M development. Monitoring Resource utilization using Grafana Tool
• Good skillset to create automation scripts in Jenkins & knowledge in working on automating builds, test frameworks, app configuration etc
• Experience in implementing scalable applications with fully automated deployment and control using Bitbucket, Jenkins, ADO etc
• Skillset: o Mandatory: Big Data – Hive, HDFS, Spark, Scala, Pyspark.
  o Good to have : Schedulers (Control-M), ETL Tool – Dataiku , Unix/Shell scripting , Knowledge of Integration services of FileIT/MQ and others , CI/CD tools – Jenkins, Jira, ADO Devops – tools suite , Oracle .

Confidentiality is assured, and only shortlisted candidates will be notified for interviews.