Lead Data Engineer (Teradata/Hadoop)

Optimum Solutions (Singapore) Pte Ltd.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Optimum Solutions (Singapore) Pte Ltd. is seeking a Senior Data Engineer with deep expertise in enterprise data platforms and banking data domains.

You will lead the design and development of robust data pipelines across Teradata, Hadoop/Cloudera, and cloud ecosystems while mentoring engineers and driving technical excellence. You will architect scalable batch, streaming, and API-based pipelines, optimise complex SQL and Spark workloads, and implement Terraform-based infrastructure with

Qualifications

• Bachelor's degree in Computer Science, Engineering, IT or related field.
• 10+ years of data-engineering experience with leadership in Teradata and Hadoop/big-data delivery.
• Certifications: Teradata, Cloudera, Databricks, Cloud Data Engineering or Terraform (preferred).
• Core tech: Teradata, Hadoop/Cloudera, Spark, Hive, Impala, HBase, SQL, Python, Scala/Java, Kafka, Airflow, Denodo.

Responsibilities

• Design and implement scalable batch, streaming, and API-based data pipelines.
• Optimize SQL queries and Spark workloads for large-volume banking data.
• Implement infrastructure-as-code with Terraform and CI/CD pipelines.
• Lead reviews, enforce standards, mentor engineers, and collaborate with architects, product teams, and DevOps.

Skills

Technical leadership

Communication

Mentoring

Education

Bachelor's degree in Computer Science or related

Tools

Teradata

Hadoop/Cloudera

Spark

Hive

Impala

HBase

SQL

Python

Scala/Java

Kafka

Airflow

Denodo

Terraform

Kubernetes/OpenShift

Git

CI/CD

Job description

About the Role

We are seeking a Senior Data Engineer with deep expertise in enterprise data platforms, banking domains, and large-scale data delivery. In this role, you will lead the design and development of robust data pipelines across Teradata, Hadoop/Cloudera, and cloud ecosystems while mentoring engineering talent and driving technical excellence.

What you’ll do

• Architecture & Development: Lead the design and implementation of scalable batch, streaming, and API-based data pipelines, historical-data processing frameworks, SCD/CDC mechanisms, and semantic-layer data products.
• Performance & Optimisation: Optimise complex SQL queries, Spark workloads, partition strategies, and storage frameworks for large-volume processing within regulated enterprise banking environments.
• Infrastructure & Automation: Implement infrastructure-as-code using Terraform and establish automated CI/CD pipelines, containerised deployments, and rigorous testing standards.
• Leadership & Collaboration: Drive code reviews, enforce engineering standards, mentor junior engineers, and partner closely with architects, product teams, DevOps, and business stakeholders.

What You Will Need to Succeed

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related discipline
• At least 10 years of data-engineering experience, with technical leadership in Teradata and Hadoop/big-data delivery.
• Preferred Certifications: Teradata, Cloudera, Databricks, Cloud Data Engineering, or Terraform
• Core Technologies: Teradata, Hadoop/Cloudera, Spark, Hive, Impala, HBase, SQL, Python, Scala/Java, Kafka, Airflow, and Denodo.
• Infrastructure & DevOps: Terraform, Kubernetes/OpenShift, Git, and CI/CD automation tools.
• Domain Expertise: Enterprise banking data (Customer, Payments, Risk, Finance, AML, and Regulatory Reporting) with a strong focus on data quality, reconciliation, lineage, and historical reconstruction.
• Core Competencies: Demonstrated technical leadership, disciplined delivery in regulated settings, exceptional communication skills, and accountability for production-ready outcomes.