Data Engineer

Alphv Recruit; Selangor; Vor Ort; MYR 120.000 - 210.000; Vollzeit

Zusammenfassung

Alphv Recruit in Malaysia is seeking a mid to senior data engineer to design scalable lakehouse architectures and data pipelines for a healthcare project. Based in Petaling Jaya, you will build ETL/ELT workflows and optimize storage while supporting ML workflows.

Join a project focused on healthcare data, with an emphasis on data privacy and reliable, real-time processing. This onsite role offers opportunities to advance in cloud and big data technologies.

Qualifikationen

• Proficient in Python, Scala, Java and SQL for data engineering tasks.
• Experience with big data frameworks and lakehouse tech stacks.
• Strong scripting skills for automation and data ops.

Aufgaben

• Design scalable data lakehouse architectures for large datasets.
• Build and maintain ETL/ELT pipelines for batch and real-time processing.
• Develop data ingestion frameworks from APIs, databases, files and streams.
• Optimize storage, partitioning and query performance in lakehouse environments.
• Support ML workflows and model deployment pipelines.
• Design disaster recovery and data backup strategies for critical assets.

Kenntnisse

Python

Scala

Java

SQL

Shell scripting

Tools

Apache Spark

Hadoop

Delta Lake

Apache Iceberg

Databricks

Snowflake

Apache Airflow

dbt

Jobbeschreibung

• A Malaysian ICT professional services and systems integration company established on 22 March 1999.
• 25+ years of experience delivering enterprise ICT and systems integration solutions across Malaysia.
• Exposure to enterprise-scale digital transformation projects, including cloud, cybersecurity, networking, infrastructure, and managed services.

Project/Goal:

• Working in client's office, serving current project in the healthcare industry

Key Responsibilities:

• Design and implement scalable data lakehouse architectures using modern technologies.
• Build and maintain ETL/ELT pipelines for batch and real-time data processing
• Develop data ingestion frameworks from various sources (APIs, databases, files, streams)
• Optimize data storage, partitioning, and query performance in lakehouse environments
• Support machine learning workflows and AI model deployment pipelines
• Design disaster recovery and backup strategies for critical data assets

Skills Requirements:

• Programming: Python, Scala, Java, SQL, Shell scripting
• Big Data: Apache Spark, Hadoop ecosystem, Delta Lake, Apache Iceberg
• Data Platforms: Databricks, Snowflake, Apache Airflow, dbt

Good to Have:

• Experience with real-time streaming platforms (Kafka, Kinesis, Pub/Sub)
• Experience with MLOps and ML pipeline orchestration
• Understanding of data privacy regulations (GDPR, Malaysian PDPA)
• Previous experience in financial services, e-commerce, or technology sectors

Work Location:

• Petaling Jaya, onsite
• Mid to Senior level with 4-6 years of experience.