Senior Big Data Platform Engineer – Spark, Flink & Kafka

CEKAP LINK TECH SDN. BHD.; Kuala Lumpur; On-site; MYR 120,000 - 180,000; Full time

Job summary

CEKAP LINK TECH SDN. BHD. in Kuala Lumpur seeks a senior big data engineer to design, develop and implement scalable data platforms. You will work with Lambda and Kappa architectures, optimize Spark and Flink, and ensure reliable data pipelines while collaborating with data analysts and engineers.

The role requires strong programming skills in SQL, Python/Java/Scala and experience with Hive and Doris/StarRocks.

Qualifications

• Bachelor's degree or higher in CS, SWE, IT or related fields.
• 3+ years of hands-on big data development experience.
• Experience with 50TB+ data volumes preferred.
• Strong knowledge of Hadoop ecosystem (Spark, Flink).
• Hands-on with streaming tech such as Kafka or Pulsar.
• Proficient in SQL, Hive and Python/Java/Scala.
• Familiar with Doris or StarRocks and performance tuning.

Responsibilities

• Design, develop and implement scalable big data platforms.
• Work with Lambda and Kappa architectures.
• Optimize Spark performance, Flink, memory management.
• Develop core modules and troubleshoot data pipelines.
• Collaborate with analysts and engineers to provide data interfaces.
• Prepare design and development documentation.

Skills

Big data development

Spark

Flink

Kafka/Pulsar

SQL

Python/Java/Scala

Hive

Doris/StarRocks

Education

Bachelor's degree or above in Computer Science, Software Engineering, Information Technology or related disciplines

Tools

Hadoop ecosystem

Hive

Doris/StarRocks

Kafka

Pulsar

Iceberg/Delta Lake

Kubernetes

EMR/CDH

Job description

CEKAP LINK TECH SDN. BHD. in Kuala Lumpur seeks a senior big data engineer to design, develop and implement scalable data platforms. You will work with Lambda and Kappa architectures, optimize Spark and Flink, and ensure reliable data pipelines while collaborating with data analysts and engineers.

The role requires strong programming skills in SQL, Python/Java/Scala and experience with Hive and Doris/StarRocks.