Data & Analytics Tech Lead: Scalable Pipelines & ETL

RISKDATA CONSULTING PTE. LTD.; Singapore; On-site; SGD 90,000 - 120,000; Full time

Job summary

Riskdata Consulting Pte. Ltd. is seeking a Data Engineer to design, develop and support enterprise-scale data engineering solutions.

You will focus on data pipelines, ETL processing, Big Data platforms and data migration across banking and financial services environments. You will design and implement data migration and modernisation solutions, develop data processing with Python, PySpark, SQL and Hive, and help optimise batch jobs and data quality.

Qualifications

• Bachelor degree or equivalent in CS/IT/related field.
• Strong hands-on data engineering with ETL, warehousing and Big Data tech.
• Proficient in Python, PySpark, SQL and Hive/HQL.
• Experience with Hadoop, Spark and enterprise data platforms.
• Experience with Talend, DataStage and/or Informatica.
• Knowledge of Teradata, Oracle or SQL Server.
• Unix/Linux and shell scripting.
• CI/CD tools like Git and Jenkins.

Responsibilities

• Design, develop and optimise scalable data pipelines and ETL processes.
• Develop data processing solutions using Python, PySpark, SQL, Hive and Spark.
• Work with Big Data platforms including Hadoop HDFS, Hive and cloud storage.
• Perform data extraction, transformation and loading from multiple source systems.
• Design and implement data migration and modernisation with DataStage, Talend, Informatica and Hadoop.
• Develop and optimise batch processing and scheduling workflows (Autosys, Control-M).
• Perform data profiling, data quality checks and performance optimisation.
• Support data warehouse and dimensional data modelling activities.
• Implement CI/CD processes for data pipeline deployments.
• Collaborate with business, application and infrastructure teams to deliver data solutions.
• Provide technical leadership and production support for enterprise data platforms.

Skills

Python

PySpark

SQL

Hive/HQL

Big Data

Data modelling

Data quality

CI/CD

Git

Jenkins

Unix/Linux

Shell scripting

ETL

Data warehousing

Talend

DataStage

Informatica

Teradata

Oracle

SQL Server

Hadoop

Spark

Autosys

Control-M

Snowflake

Cloudera

Education

Degree in Computer Science or related

Tools

Talend

DataStage

Informatica

Hadoop

Hive

Job description

Riskdata Consulting Pte. Ltd. is seeking a Data Engineer to design, develop and support enterprise-scale data engineering solutions.

You will focus on data pipelines, ETL processing, Big Data platforms and data migration across banking and financial services environments. You will design and implement data migration and modernisation solutions, develop data processing with Python, PySpark, SQL and Hive, and help optimise batch jobs and data quality.