BigData Engineer

NTT Data Singapore; Singapore; On-site; SGD 95,000 - 150,000; Full time

Job summary

NTT Data Singapore is seeking a skilled Big Data engineer with 5+ years of experience to develop and maintain enterprise-class data applications in a banking environment.

The role requires deep expertise in Hadoop, Spark, Hive, and related tools, plus strong communication and collaboration across cross-functional teams. Proficiency in CI/CD, Linux shell scripting, and Agile practices is essential.

Qualifications

• University graduate in Computer Science or related discipline.
• 5-7 years of experience in developing, maintaining and supporting enterprise class big data applications, preferably in banking.
• Proficient in big data technologies/tools/platforms such as Hadoop, HDFS, Hive, Impala, Presto, Spark, Zeppelin, Yarn, Cloudera, Hortonworks.
• Experience with relational databases and Spark performance tuning; Java Spark preferred but not mandatory.
• Knowledge of CI/CD with Jenkins, Git and Bitbucket; Linux shell scripting.
• Familiar with Parquet/ORC/AVRO/JSON formats and microservices architecture.
• Strong communication, analytical skills, and experience in Agile environments.

Responsibilities

• Develop and maintain enterprise-grade big data applications.
• Tune Spark jobs for performance and reliability.
• Collaborate with cross-functional and cross-regional teams.
• Adhere to best practices for Spark programming and data processing.
• Participate in CI/CD pipelines and code reviews.

Skills

Big Data

Spark

Hadoop

Hive

Python/Java

CI/CD

Linux Shell

Communication

Agile

Education

University Graduate in Computer Science or Related discipline

Tools

Hadoop

Spark

Hive

Presto

Zeppelin

Yarn

Cloudera

Hortonworks

Jenkins

Git

Bitbucket

Parquet/ORC/AVRO/JSON

S3

Job description

• University Graduate in Computer Science or Related discipline
• 5-7 years of experience in developing, maintaining and supporting Enterprise Class Big Data Applications, preferably in banking environments
• Expertise in Big Data technologies/tools/platforms such as Hadoop, HDFS, Hive, Impala, HDFS, Presto, Spark, Hive, Impala, Zeppelin, Yarn, Cloudera, Hortonworks
• Experience in Relational Databases
• Knowledge and experience in coding and performance tuning of Spark jobs. Java Spark preferred but not compulsory
• Establish and adhere to best practices relating to Apache Spark programming
• Responsible for understanding functional/non-functional requirements and implement them in Apache Spark
• Experience in usage of CI/CD tools Jenkins, Git, Bitbucket
• Experience in writing Linux basic Shell Scripts
• Knowledge and experience in Apache Spark batch and streaming framework
• Knowledge of working with different file formats like Parquet, ORC, AVRO and JSON
• Knowledge of Micro Services architecture
• Knowledge of S3 is added advantage
• Ability to work proactively, independently and with cross-functional and cross-regional teams
• Strong communication and analytical skills and experience in Agile projects