Senior Data Engineer: Real-Time ETL & Data Architecture

Alliance Bank Malaysia Berhad; Kuala Lumpur; On-site; MYR 120,000 - 180,000; Full time

Job summary

Alliance Bank Malaysia Berhad is seeking a data engineer to design, build, and maintain scalable ETL/ELT pipelines for batch and real-time data processing. You will integrate data from multiple internal and external sources to support analytics and reporting.

You will develop data models, ensure data quality, automate workflows with Airflow, and collaborate with analysts, data scientists, and stakeholders across cloud platforms (AWS/Azure/GCP).

Qualifications

• Bachelor's degree in CS/Engineering/IS or related field.
• Experience with real-time data streaming and processing.
• Familiarity with Docker and Kubernetes.
• Knowledge of data modeling techniques (dimensional, star schema).
• Experience with CI/CD pipelines for data workflows.
• Understanding of machine learning workflows and data preparation.
• Strong programming skills in Python, Java, or Scala.
• Experience with SQL and NoSQL databases.
• Hands-on experience with data pipeline tools (Airflow, Talend, Informatica).
• Familiarity with Hadoop, Spark, Kafka.
• Experience with cloud platforms such as AWS, Azure, or Google Cloud.
• Understanding of data warehousing concepts (Snowflake, Redshift, BigQuery).
• Knowledge of version control systems (Git).

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines for batch and real-time data processing.
• Integrate data from multiple sources (databases, APIs, streaming platforms, third-party systems).
• Ensure efficient data ingestion, transformation, and loading processes.
• Develop and maintain data models (dimensional models, star/snowflake schemas).
• Transform raw data into analytics-ready datasets.
• Optimize data structures for performance, scalability, and usability.
• Implement data validation, cleansing, and enrichment processes.
• Establish data quality frameworks, including rules, checks, and monitoring.
• Troubleshoot data inconsistencies and failures; ensure high availability.
• Work with cloud data platforms (Azure, AWS, GCP) and data services (Synapse, BigQuery, Redshift).
• Monitor pipeline performance and improve throughput and latency.
• Collaborate with analysts, data scientists, architects, and stakeholders.
• Provide clean datasets for reporting, dashboards, and ML use cases.
• Support self-service analytics with easy data access and documentation.
• Automate data workflows using orchestration tools (Airflow, Azure Data Factory).
• Apply version control and infrastructure-as-code practices.
• Promote reusable components and standardized frameworks.
• Set up monitoring, alerting, and logging.

Skills

Python

Java

Scala

SQL

NoSQL

Real-time data streaming

Data modeling

Data warehousing

Education

Bachelor's degree in Computer Science, Engineering, Information Systems, or related field

Tools

Docker

Kubernetes

Apache Airflow

Talend

Informatica

Hadoop

Spark

Kafka

AWS

Azure

Google Cloud

Snowflake

Redshift

BigQuery

CI/CD

Git

Job description

Alliance Bank Malaysia Berhad is seeking a data engineer to design, build, and maintain scalable ETL/ELT pipelines for batch and real-time data processing. You will integrate data from multiple internal and external sources to support analytics and reporting.

You will develop data models, ensure data quality, automate workflows with Airflow, and collaborate with analysts, data scientists, and stakeholders across cloud platforms (AWS/Azure/GCP).