Lead Big Data & Analytics Engineer (Flexible Hours)

Sunway Digital Hub, Digital Technology Solutions & Group Cybersecurity; Selangor; On-site; MYR 120,000 - 180,000; Full time

Benefits offered by this job

Group medical insurance (outpatient &&

Dental and optical benefits

Staff discounts

Flexible working hours

Job summary

Sunway Digital Hub, Digital Technology Solutions & Group Cybersecurity – Petaling is seeking a data professional to design and optimize large-scale data platforms. You will develop data processing pipelines using Spark, Hadoop, and Kafka, ensuring data quality and scalable architectures.

Requirements include experience with Python, SQL, Terraform, Docker, and familiarity with Huawei Cloud DLI, Databricks, Airflow, and data warehousing concepts.

Qualifications

• Experience with big data technologies (Spark preferred, Hadoop or Kafka).
• Strong programming skills: Python, SQL, Terraform, Docker.
• Experience with big data analytics platforms (Huawei Cloud DLI, Databricks, AWS EMR).
• Familiarity with data warehousing concepts (Huawei DWS, Airflow, BigQuery, Redshift, Snowflake).
• Knowledge of data modelling (supervised & unsupervised).
• Experience with data visualization tools (PowerBI preferred, Tableau, QuickSight).
• Experience with MSSQL, Postgres, and NoSQL databases.
• Strong problem-solving and attention to detail.

Responsibilities

• Design, build, and maintain big data systems with stakeholder requirements.
• Develop data processing solutions using Spark, Hadoop, Kafka ensuring data accuracy.
• Apply advanced data science techniques (predictive modeling, NLP) to build data products.
• Collaborate with marketing, engineering, product to translate business needs into technical specs.
• Ensure compliance with data privacy and security regulations.
• Stay updated with data science and martech trends to inform solutions.

Skills

Big data technologies

Python

SQL

Terraform

Docker

Pyspark

Huawei Cloud (DLI)

Databricks

Airflow

PowerBI

Education

Bachelor’s degree in a relevant field

Tools

Hadoop

Spark

Kafka

MSSQL

Postgres

NoSQL

Job description

Sunway Digital Hub, Digital Technology Solutions & Group Cybersecurity – Petaling is seeking a data professional to design and optimize large-scale data platforms. You will develop data processing pipelines using Spark, Hadoop, and Kafka, ensuring data quality and scalable architectures.

Requirements include experience with Python, SQL, Terraform, Docker, and familiarity with Huawei Cloud DLI, Databricks, Airflow, and data warehousing concepts.