Senior Data Engineer: Hadoop Data Lake & Spark Solutions

D L RESOURCES PTE LTD; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

D L RESOURCES PTE LTD in Singapore seeks a skilled data engineer to design, develop and optimize large-scale data pipelines using Hadoop and Spark. You will work with data lake architecture and NoSQL/SQL data stores to ingest, transform and load data for downstream analytics.

The role requires collaboration with data scientists and engineers, ensuring data quality and performance across SIT, UAT, PROD environments, with a focus on banking data integration.

Qualifications

• Strong experience in developing Hadoop/Spark.
• Experience in data lakes and integrating diverse data sources.
• Prior ETL experience in banking applications is mandatory.
• Deep knowledge of technology stacks used by global banks.

Responsibilities

• Design, develop, and implement data processing pipelines for large data volumes.
• Work with Hadoop, Hive, Impala, and Kudu for data storage and access.
• Script using Unix shell, awk, and automation to integrate third-party tools.
• Model data using industry standard models like FSLDM.
• Collaborate with data engineers, data scientists, and stakeholders to translate requirements.
• Monitor job performance and optimize Spark/Elasticsearch integration.

Skills

Hadoop

Data Lake

SQL

Cloudera

Informatica

Tools

Hive

Impala

Kudu

Job description

D L RESOURCES PTE LTD in Singapore seeks a skilled data engineer to design, develop and optimize large-scale data pipelines using Hadoop and Spark. You will work with data lake architecture and NoSQL/SQL data stores to ingest, transform and load data for downstream analytics.

The role requires collaboration with data scientists and engineers, ensuring data quality and performance across SIT, UAT, PROD environments, with a focus on banking data integration.