Senior Data Engineering Lead - Pipelines & Modernisation

regtech insight pte. ltd.; Singapore; On-site; SGD 120,000 - 160,000; Full time

Job summary

regtech insight pte. ltd. is seeking a Data Engineer to design, develop and support enterprise-scale data solutions for banking and financial services environments.

You will build and optimise scalable data pipelines using Python, PySpark, SQL and Hive, work with Hadoop and cloud storage, and ensure data quality and governance across complex data landscapes. Collaborate with cross-functional teams, implement CI/CD for data pipelines and deliver production-ready data platforms with strong

Qualifications

• Degree in CS/IT/Data Eng or related discipline.
• Strong experience in data engineering, ETL, data warehousing and Big Data.
• Hands-on with Python, PySpark, SQL and Hive/HQL.
• Experience with Hadoop, Spark and enterprise data platforms.
• Experience with Talend, DataStage and/or Informatica.
• Experience with Teradata, Oracle or SQL Server.
• Unix/Linux and Shell scripting experience.
• CI/CD tools such as Git and Jenkins.
• Strong data modelling, data quality and pipeline optimisation.
• Banking/financial services experience is a plus.

Responsibilities

• Design, develop and optimise scalable data pipelines and ETL.
• Develop and maintain data processing using Python, PySpark, SQL, Hive and Spark.
• Work with Big Data platforms including Hadoop HDFS and Hive.
• Perform data extraction, transformation and loading from multiple sources.
• Design and implement data migration and modernisation solutions.
• Develop and optimise batch processing and scheduling workflows.
• Perform data profiling and data quality checks.
• Support data warehouse and dimensional modelling activities.
• Implement CI/CD processes for data pipeline deployments.
• Collaborate with business, application and infra teams.
• Provide technical leadership and production support.

Skills

Python

PySpark

SQL

Hive

Hadoop

Git

Jenkins

Data modelling

Data quality

Data pipeline

CI/CD

Spark

UNIX/Linux

Shell scripting

Talend

DataStage

Informatica

Teradata

Oracle

SQL Server

Education

Bachelor's degree in CS/IT/Data Eng

Tools

DataStage

Talend

Informatica

Hadoop

Snowflake

Cloudera

Autosys

Control-M

Job description

regtech insight pte. ltd. is seeking a Data Engineer to design, develop and support enterprise-scale data solutions for banking and financial services environments.

You will build and optimise scalable data pipelines using Python, PySpark, SQL and Hive, work with Hadoop and cloud storage, and ensure data quality and governance across complex data landscapes. Collaborate with cross-functional teams, implement CI/CD for data pipelines and deliver production-ready data platforms with strong