Data Platform Engineer

Alliance Bank Malaysia Berhad; Kuala Lumpur; On-site; MYR 120,000 - 210,000; Full time

Job summary

Alliance Bank Malaysia Berhad in Kuala Lumpur seeks a seasoned data platform engineer to design scalable data lakes, lakehouse, and warehouses, and to build distributed systems that support batch, streaming, and real-time processing. You will implement IaC, ensure governance, security, and observability, and collaborate with data scientists and engineers to deliver reusable services.

Join a team focusing on self-service analytics, data quality, and cost-efficient operations across Azure, AWS,

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• 5+ years of experience in data engineering, platform engineering, or related roles.
• Experience with AI/ML data pipelines and feature stores.
• Knowledge of data security frameworks and zero-trust architecture.
• Familiarity with FinOps practices for data platform cost optimization.

Responsibilities

• Design and implement scalable, resilient, and secure data platforms (data lakes, lakehouse, data warehouses).
• Build and maintain distributed data systems that support batch, streaming, and real-time processing.
• Develop reusable frameworks and platform services to standardize data ingestion, processing, and access.
• Build and maintain robust ETL/ELT pipelines using modern orchestration tools.
• Ensure data quality, lineage, observability, and governance are embedded within pipelines.
• Optimize data workflows for performance, cost efficiency, and reliability.
• Deploy, manage, and optimize data platforms on cloud providers (Azure, AWS, GCP).
• Implement Infrastructure as Code (IaC) using tools like Terraform, ARM/Bicep, or CloudFormation.
• Monitor and manage platform performance, availability, and scalability.
• Implement data governance frameworks, including cataloging, classification, and lineage tracking.

Skills

Python

Java

Scala

SQL

Data modeling

Streaming analytics

Cloud platforms

Security best practices

Education

Bachelor’s or Master’s degree in Computer Science, Engineering, or related field

Tools

Airflow

Azure Data Factory

Prefect

Dagster

Docket

Git

Terraform

Docker

Kubernetes

GitHub Actions

Jenkins

BigQuery

Dataflow

Managed Spark

Hadoop

Job description

• Design and implement scalable, resilient, and secure data platforms (data lakes, lakehouse, data warehouses).
• Build and maintain distributed data systems that support batch, streaming, and real-time processing.
• Develop reusable frameworks and platform services to standardize data ingestion, processing, and access.
• Build and maintain robust ETL/ELT pipelines using modern orchestration tools.
• Ensure data quality, lineage, observability, and governance are embedded within pipelines.
• Optimize data workflows for performance, cost efficiency, and reliability.
• Deploy, manage, and optimize data platforms on cloud providers (Azure, AWS, GCP).
• Implement Infrastructure as Code (IaC) using tools like Terraform, ARM/Bicep, or CloudFormation.
• Monitor and manage platform performance, availability, and scalability.

Data Governance & Security

• Implement data governance frameworks, including cataloging, classification, and lineage tracking.
• Ensure compliance with data security and privacy standards (e.g., GDPR, PDPA).
• Manage access control, encryption, and auditing mechanisms.

DevOps & Automation

• Build CI/CD pipelines for data platform components.
• Automate deployments, monitoring, and alerting.
• Apply SRE principles to improve platform reliability and availability.
• Partner with data engineers, data scientists, and business stakeholders to deliver data solutions.
• Provide platform best practices and guidelines to engineering teams.
• Support self-service data capabilities for analytics and AI use cases.

Requirement

Education & Experience

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• 5+ years of experience in data engineering, platform engineering, or related roles.
• Experience with AI/ML data pipelines and feature stores.
• Knowledge of data security frameworks and zero-trust architecture.
• Familiarity with FinOps practices for data platform cost optimization.

Technical Skills

• Strong programming skills (Python, Java, Scala, or similar).
• Experience with distributed data processing frameworks (Spark, Flink, or equivalent).
• Proficiency in SQL and data modeling techniques.
• Hands-on experience with cloud data services:
• AWS: S3, Glue, Redshift, EMR
• GCP: BigQuery, Dataflow, Composer, Managed Spark, Hadoop
• Experience with modern data architectures (Lakehouse, Data Mesh, Data Fabric).
• Tools such as Airflow, Azure Data Factory, Prefect, or Dagster, Docket, Git.
• Experience with streaming platforms (Kafka, Redpanda Event Hubs, Kinesis).

DevOps & Infrastructure

• Familiarity with containerization (Docker) and orchestration (Kubernetes).
• Experience with CI/CD tools (Azure DevOps, GitHub Actions, Jenkins).
• Infrastructure as Code (Terraform preferred).

Data Governance, Security & Observability

• Experience with tools like Collibra, Purview, DataHub, Prometheus, Grafana, OpenLineage/ Apache Ranger
• Understanding data quality frameworks and monitoring tools.