Data Engineer

Itmax System Berhad; Selangor; On-site; MYR 60,000 - 80,000; Full time

Job summary

A technology company in Selangor is seeking a Data Engineer to design, build, and operate data pipelines using Apache Spark and Kafka. The role involves data ingestion, transformation, and close collaboration with Platform Integration Engineers for infrastructure management. Ideal candidates should have strong programming skills in Python, Java, or Scala, and hands-on experience with Spark and Kafka. The position offers a clear growth path towards becoming a Senior Data Engineer within a dedicated engineering team.

Benefits

Ownership of core data pipelines

Hands-on experience with cutting-edge technologies

Clear growth path to Senior Data Engineer

Qualifications

• Strong experience in programming languages: Python, Java, or Scala.
• Hands-on experience with Apache Spark and data pipeline design.
• Working knowledge of SQL, PostgreSQL, and Linux environments.

Responsibilities

• Design and build large-scale batch processing pipelines.
• Develop ETL/ELT workflows for analytics and reporting.
• Collaborate on deployment and reliability with integration teams.

Skills

Programming in Python, Java, or Scala

Apache Spark

SQL and PostgreSQL

Apache Kafka

Linux

Tools

Docker

Kubernetes

Job description

We are hiring a Data Engineer to design, build, and operate batch and event-driven data pipelines on a modern on-premise data platform.

This role focuses on data ingestion, transformation, and processing, using Apache Spark and Kafka, supporting analytics, reporting, and operational dashboards. You will work closely with Platform Integration Engineers, who manage the underlying infrastructure and streaming platform.

Apache Spark (SQL / PySpark / Structured Processing), Apache Kafka, Batch & Streaming Data Pipelines, ETL / ELT, CDC (Change Data Capture), PostgreSQL / Relational Databases, Docker / Kubernetes, Linux, On-Prem Data Platform

Key Responsibilities

• Design, build, and operate large-scale batch processing pipelines using Apache Spark
• Develop ETL / ELT workflows for analytics and reporting
• Implement data ingestion pipelines from databases, APIs, and event sources
• Optimise data pipelines for performance, reliability, resource efficiency and data quality

Streaming & Event-Driven Data

• Consume and process events from Kafka
• Introduce streaming or near-real-time processing when required
• Work with CDC pipelines and event-based data sources
• Support schema evolution and downstream data consumption

Collaboration & Operations

• Collaborate with Platform Integration Engineers on deployment, observability, and reliability
• Work with backend and application teams on data requirements and integration
• Participate in troubleshooting data pipeline issues and performance bottlenecks
• Maintain documentation, data models, and design decisions

Requirements

We do not expect every candidate to meet every requirement, but strong experience in most of the following is important.

Core Skills

• Strong programming skills in Python, Java, or Scala
• Hands-on experience with Apache Spark
• Working knowledge of Apache Kafka or other messaging platforms
• Strong SQL skills and experience with PostgreSQL or other RDBMS
• Familiarity with Linux environments

Nice to Have

• Experience with Apache Flink, Spark Streaming, Kafka Streams
• Exposure to CDC tools (Debezium, Maxwell, GoldenGate)
• Experience with Docker and Kubernetes
• Familiarity with workflow orchestration tools (Airflow or similar)
• Experience with monitoring, CI/CD, or infrastructure-as-code tools

Who Should Apply

• Data Engineers who enjoy pipeline design and optimisation
• Backend engineers moving into data engineering
• Candidates interested in batch and streaming data systems
• Strong fundamentals with willingness to learn are welcome

What We Offer

• Ownership of core data pipelines in a production system
• Hands-on experience with Spark, Kafka, and distributed data systems
• Clear growth path toward Senior Data Engineer / Data Architect
• High-impact role in a small, engineering-focused team