Senior Data Platform Developer - Hadoop, Spark & AWS

DBS; Singapore; On-site; SGD 120,000 - 160,000; Full time

Job summary

DBS invites applications for Senior Assoc, Application Developer in Middle Office Technology, Finance Platform in Singapore. You will design, develop, and maintain scalable data pipelines within the Hadoop ecosystem and optimize Spark/SQL performance to reduce latency.

You will automate batch processes using Shell scripts and REST APIs, manage data storage with HDFS/Parquet, and leverage AWS S3 for cloud data management. Strong Linux/Unix, Python, and Scala skills are required.

Qualifications

• Experience designing and maintaining scalable data pipelines in a Hadoop ecosystem.
• Proficient in tuning SQL and Spark ETL for performance.
• Experience with batch automation, REST APIs, and Linux scripting.
• Cloud experience with AWS S3 preferred.

Responsibilities

• Design, develop, and maintain data pipelines and apps within Hadoop.
• Tune SQL and Spark ETL to optimize latency.
• Automate batch processes using Shell scripts and REST APIs.
• Manage data storage with HDFS and Parquet.
• Develop scripts in Shell, Python, and Scala for automation.
• Troubleshoot issues in Linux/Unix environments.

Skills

Analytical thinking

Problem solving

Team collaboration

Communication

Tools

Hadoop

HDFS

Hive

Impala

Spark (Scala/Java)

SQL

Shell scripting

Python

Scala

Linux/Unix

REST APIs

AWS S3

Job description

DBS invites applications for Senior Assoc, Application Developer in Middle Office Technology, Finance Platform in Singapore. You will design, develop, and maintain scalable data pipelines within the Hadoop ecosystem and optimize Spark/SQL performance to reduce latency.

You will automate batch processes using Shell scripts and REST APIs, manage data storage with HDFS/Parquet, and leverage AWS S3 for cloud data management. Strong Linux/Unix, Python, and Scala skills are required.