Senior Data Engineer: Big Data Pipelines & ETL Lead

RISKDATA CONSULTING PTE. LTD.; Singapore; On-site; SGD 70,000 - 110,000; Full time

Job summary

Riskdata Consulting Pte. Ltd. is seeking a Data Engineer to design, develop and support enterprise-scale data solutions in Singapore.

You will focus on building data pipelines, ETL processes and data quality across banking and financial services environments. You will work with Python, PySpark, SQL and Hive to implement scalable data platforms, perform data migration, profiling and optimisation, and participate in CI/CD for data pipelines while collaborating with business and infrastructure

Qualifications

• Degree in Computer Science, Information Technology, Data Engineering or a related discipline.
• Strong hands‑on experience in Python, PySpark, SQL and Hive/HQL.
• Experience with Hadoop, Spark and enterprise data platforms.
• Experience with Talend, DataStage and/or Informatica.
• Experience with Teradata, Oracle or SQL Server.
• Experience with Unix/Linux and Shell scripting.
• Experience with CI/CD tools such as Git and Jenkins.
• Strong understanding of data modelling, data quality and data pipeline optimisation.
• Banking or financial services experience will be an advantage.

Responsibilities

• Design, develop and optimise scalable data pipelines and ETL processes.
• Develop and maintain data processing solutions using Python, PySpark, SQL, Hive and Spark.
• Work with Big Data platforms including Hadoop HDFS, Hive and cloud/object storage environments.
• Perform data extraction, transformation and loading from multiple source systems.
• Design and implement data migration and modernisation solutions involving DataStage, Talend, Informatica and Hadoop.
• Develop and optimise batch processing and scheduling workflows using Autosys and Control-M.
• Perform data profiling, data quality checks and performance optimisation.
• Support data warehouse and dimensional data modelling activities.
• Implement CI/CD processes for data pipeline deployments.
• Work with business, application and infrastructure teams to deliver data solutions.
• Provide technical leadership and production support for enterprise data platforms.

Skills

Python

PySpark

SQL

Hive

Shell scripting

Education

Bachelor's degree in Computer Science or related field

Tools

Hadoop

Spark

DataStage

Talend

Informatica

Git

Jenkins

Job description

Riskdata Consulting Pte. Ltd. is seeking a Data Engineer to design, develop and support enterprise-scale data solutions in Singapore.

You will focus on building data pipelines, ETL processes and data quality across banking and financial services environments. You will work with Python, PySpark, SQL and Hive to implement scalable data platforms, perform data migration, profiling and optimisation, and participate in CI/CD for data pipelines while collaborating with business and infrastructure