AI-Driven Platform Data Engineer

NCS Group; Kuala Lumpur; On-site; MYR 150,000 - 200,000; Full time

Job summary

A leading technology services firm in Kuala Lumpur seeks a Platform Data Engineer to design and operate data pipelines for AI workloads. The ideal candidate will have strong data processing skills, experience with batch and stream processing technologies, and familiarity with event-driven architectures. This role embodies a collaborative spirit, values reliability, and promotes excellence through innovative data solutions. Join a diverse team committed to making extraordinary things happen.

Qualifications

• Strong data processing fundamentals for AI and ML pipelines.
• Hands-on experience with batch processing frameworks (e.g., Apache Spark or equivalent).
• Experience with stream processing technologies (e.g., Kafka Streams).
• Familiarity with event-driven architectures using platforms such as Kafka.
• Hands-on experience with relational databases (e.g., Oracle Autonomous Database).
• Experience managing stateful services and caches (e.g., Redis Enterprise).
• Familiarity with vector databases (e.g., Milvus).
• Experience operating data pipelines in containerized environments.
• Understanding of data observability, data quality checks.
• Strong systems-thinking and data modeling skills.

Responsibilities

• Design and build scalable batch and streaming data pipelines for AI and ML workloads.
• Enable real-time and near-real-time data ingestion to support AI inference.
• Ensure data pipelines are reliable, observable, and production-ready.
• Collaborate with AI engineers and platform teams on data contracts.
• Manage stateful services, caches, and support vector databases.

Skills

Data processing fundamentals for AI and ML pipelines

Experience with batch processing frameworks (e.g., Apache Spark)

Experience with stream processing technologies (e.g., Kafka Streams)

Familiarity with event-driven architectures

Hands-on experience with relational databases

Experience managing stateful services and caches

Familiarity with vector databases

Experience operating data pipelines in containerized environments

Understanding of data observability and quality checks

Strong systems-thinking and data modeling skills

Ownership mentality for long-lived data pipelines

Job description

A leading technology services firm in Kuala Lumpur seeks a Platform Data Engineer to design and operate data pipelines for AI workloads. The ideal candidate will have strong data processing skills, experience with batch and stream processing technologies, and familiarity with event-driven architectures. This role embodies a collaborative spirit, values reliability, and promotes excellence through innovative data solutions. Join a diverse team committed to making extraordinary things happen.