BigData Engineer

NTT Data Singapore; Singapore; On-site; SGD 70,000 - 100,000; Full time

Job summary

A leading Big Data solutions provider in Singapore seeks an experienced big data engineer. The role requires developing and supporting enterprise-level applications using technologies like Hadoop and Spark. Candidates should have 5-7 years of experience in banking environments, a computer science background, and skills in CI/CD tools. Strong analytical and communication skills are essential, along with the ability to work across teams in an agile environment.

Qualifications

• 5-7 years of experience in developing, maintaining and supporting Enterprise Class Big Data Applications.
• Knowledge and experience in coding and performance tuning of Spark jobs.
• Experience in usage of CI/CD tools like Jenkins, Git, and Bitbucket.

Responsibilities

• Understand functional/non-functional requirements and implement them in Apache Spark.
• Establish and adhere to best practices relating to Apache Spark programming.
• Work proactively and independently with cross-functional and cross-regional teams.

Skills

Big Data technologies/tools/platforms

Java Spark

CI/CD tools (Jenkins, Git, Bitbucket)

Linux Shell scripting

Agile projects

Micro Services architecture

Communication skills

Analytical skills

Education

University Graduate in Computer Science or Related discipline

Tools

Hadoop

HDFS

Hive

Spark

Zeppelin

Cloudera

Hortonworks

Presto

Job description

• University Graduate in Computer Science or Related discipline
• 5-7 years of experience in developing, maintaining and supporting Enterprise Class Big Data Applications, preferably in banking environments
• Expertise in Big Data technologies/tools/platforms such as Hadoop, HDFS, Hive, Impala, HDFS, Presto, Spark, Hive, Impala, Zeppelin, Yarn, Cloudera, Hortonworks
• Experience in Relational Databases
• Knowledge and experience in coding and performance tuning of Spark jobs. Java Spark preferred but not compulsory
• Establish and adhere to best practices relating to Apache Spark programming
• Responsible for understanding functional/non-functional requirements and implement them in Apache Spark
• Experience in usage of CI/CD tools Jenkins, Git, Bitbucket
• Experience in writing Linux basic Shell Scripts
• Knowledge and experience in Apache Spark batch and streaming framework
• Knowledge of working with different file formats like Parquet, ORC, AVRO and JSON
• Knowledge of Micro Services architecture
• Knowledge of S3 is added advantage
• Ability to work proactively, independently and with cross-functional and cross-regional teams
• Strong communication and analytical skills and experience in Agile projects