Platform Data Engineer

NCS Group; Kuala Lumpur; On-site; MYR 150,000 - 200,000; Full time

Job summary

A leading technology services firm in Kuala Lumpur seeks a Platform Data Engineer to design and operate data pipelines for AI workloads. The ideal candidate will have strong data processing skills, experience with batch and stream processing technologies, and familiarity with event-driven architectures. This role embodies a collaborative spirit, values reliability, and promotes excellence through innovative data solutions. Join a diverse team committed to making extraordinary things happen.

Qualifications

• Strong data processing fundamentals for AI and ML pipelines.
• Hands-on experience with batch processing frameworks (e.g., Apache Spark or equivalent).
• Experience with stream processing technologies (e.g., Kafka Streams).
• Familiarity with event-driven architectures using platforms such as Kafka.
• Hands-on experience with relational databases (e.g., Oracle Autonomous Database).
• Experience managing stateful services and caches (e.g., Redis Enterprise).
• Familiarity with vector databases (e.g., Milvus).
• Experience operating data pipelines in containerized environments.
• Understanding of data observability, data quality checks.
• Strong systems-thinking and data modeling skills.

Responsibilities

• Design and build scalable batch and streaming data pipelines for AI and ML workloads.
• Enable real-time and near-real-time data ingestion to support AI inference.
• Ensure data pipelines are reliable, observable, and production-ready.
• Collaborate with AI engineers and platform teams on data contracts.
• Manage stateful services, caches, and support vector databases.

Skills

Data processing fundamentals for AI and ML pipelines

Experience with batch processing frameworks (e.g., Apache Spark)

Experience with stream processing technologies (e.g., Kafka Streams)

Familiarity with event-driven architectures

Hands-on experience with relational databases

Experience managing stateful services and caches

Familiarity with vector databases

Experience operating data pipelines in containerized environments

Understanding of data observability and quality checks

Strong systems-thinking and data modeling skills

Ownership mentality for long-lived data pipelines

Job description

NCS is a leading technology services firm that operates across the Asia Pacific region in over 20 cities, providing consulting, digital services, technology solutions, and more. We believe in harnessing the power of technology to achieve extraordinary things, creating lasting value and impact for our communities, partners, and people. Our diverse workforce of 14,000 has delivered large-scale, mission-critical, and multi-platform projects for governments and enterprises in Singapore and the APAC region.

Job Description

As a Platform Data Engineer, you will be part of the AI Center of Excellence (AI COE), building and operating data pipelines that power AI and ML workloads on an enterprise-grade platform. You will enable reliable, scalable data flows for model training, inference, and AI-driven applications, ensuring production-ready, observable, and maintainable data services.

Responsibilities

• Data Pipeline Development & Operations: Design and build scalable batch and streaming data pipelines for AI and ML workloads
• Enable real-time and near-real-time data ingestion to support AI inference and agent workflows
• Ensure data pipelines are reliable, observable, and production-ready
• Collaborate with AI engineers and platform teams on data contracts, standards, and integration patterns
• Contribute to platform documentation and data engineering best practices

• Data Management & Access: Support data access patterns for training, evaluation, and production inference
• Manage stateful services, caches, and support vector databases for embedding-based retrieval

Qualifications

• The ideal candidate should possess:
• Strong data processing fundamentals for AI and ML pipelines
• Hands-on experience with batch processing frameworks (e.g., Apache Spark or equivalent)
• Experience with stream processing technologies (e.g., Kafka Streams, structured streaming, or equivalent)
• Familiarity with event-driven architectures using platforms such as Kafka
• Hands-on experience with relational databases (e.g., Oracle Autonomous Database, PostgreSQL)
• Experience managing stateful services and caches (e.g., Redis Enterprise)
• Familiarity with vector databases (e.g., Milvus) and embedding-based retrieval patterns
• Experience operating data pipelines in containerized / Kubernetes-based environments
• Understanding of data observability, data quality checks, and pipeline monitoring
• Strong systems-thinking and data modeling skills
• Reliability and correctness mindset for shared data services
• Ability to collaborate across AI and platform domains
• Ownership mentality for long-lived data pipelines

Additional Information

We are driven by our AEIOU beliefs—Adventure, Excellence, Integrity, Ownership, and Unity—and we seek individuals who embody these values in both their professional and personal lives. We are committed to our Impact: Valuing our clients, Growing our people, and Creating our future. Together, we make the extraordinary happen.

Learn more about us at ncs.co and visit our LinkedIn career site.