Senior Big Data Platform Engineer - Scalable & Secure Infra

Devopsroles; Singapore; On-site; SGD 180,000 - 240,000; Full time

Job summary

CSIT, an agency under the Ministry of Defence (Singapore), seeks a Senior Big Data Platform Staff Engineer to design, implement, and optimise a robust Big Data infrastructure. You will lead initiatives to ensure high availability, scalability and performance of data processing and analytics environments.

You will work with Spark, Iceberg, Trino, Hadoop, and Kubernetes to manage large-scale data workloads, implement secure data access, and optimise pipelines.

Qualifications

• 8+ years of experience in managing Big Data Platforms.
• Proven experience with Spark, Iceberg, Trino, Elastic Stack, NiFi, Kafka, Airflow.
• Expertise in building streaming/batch pipelines and deploying AI/ML infrastructure.

Responsibilities

• Design, implement and manage the Big Data Infrastructure platform (Hadoop, Spark, NoSQL, data lakes).
• Manage Big Data clusters: monitoring, troubleshooting, and optimizing performance; manage job scheduling and resource allocation.
• Design and implement storage solutions (HDFS, S3, other object stores).
• Ensure security and governance: access controls, encryption, audits.

Education

Bachelor's degree in Computer Science or related field

Tools

Spark

Iceberg

Trino

Elastic Stack

NiFi

Kafka

Airflow

Hadoop

S3

Kubernetes

Python

Scala

Java

Bash

Job description

CSIT, an agency under the Ministry of Defence (Singapore), seeks a Senior Big Data Platform Staff Engineer to design, implement, and optimise a robust Big Data infrastructure. You will lead initiatives to ensure high availability, scalability and performance of data processing and analytics environments.

You will work with Spark, Iceberg, Trino, Hadoop, and Kubernetes to manage large-scale data workloads, implement secure data access, and optimise pipelines.