Senior Big Data Platform Engineer - Scalable Pipelines

Lever, Inc.; Singapore; On-site; SGD 140,000 - 200,000; Full time

Job summary

CSIT is seeking a Senior Big Data Platform Staff Engineer to design, build, and operate a resilient Big Data infrastructure in Singapore. You will lead the engineering of scalable data lakes, data warehouses and streaming pipelines, while ensuring high availability and performance for national security–related analytics.

You will collaborate with data engineers and scientists, manage cluster resources, and drive security, governance and CI/CD integration using Kubernetes and related tools.

Qualifications

• Bachelor’s degree or equivalent in CS/engineering.
• 8+ years managing Big Data Platforms with Spark, Iceberg, Trino, Elastic Stack, NiFi, Kafka, Airflow.
• Expertise building streaming/batch pipelines and data warehouses.
• Experience with S3/NFS/SMB/HDFS storage and data governance.
• Strong security, encryption, and AuthN/AuthZ using OIDC/OAuth2.
• Solid networking knowledge including DNS, routing, subnets.
• Proficient with Bash, Python, Scala and Java; Kubernetes.
• Experience with large-scale search workloads and version control.

Responsibilities

• Design, implement and manage the Big Data Infrastructure platform (Hadoop, Spark, NoSQL, data lakes).
• Manage Big Data clusters: monitoring, troubleshooting, resource allocation.
• Design storage solutions (HDFS, S3) for Big Data workloads.
• Ensure security governance: access controls, encryption, audits.
• Collaborate with data engineers/scientists to optimize performance.
• Plan capacity forecasting, procurement and deployment of hardware/software.

Education

Bachelor’s degree in Computer Science, Engineering, or a related field

Tools

Hadoop

Spark

Trino

Elastic Stack

NiFi

Kafka

Airflow

Kubernetes

Job description

CSIT is seeking a Senior Big Data Platform Staff Engineer to design, build, and operate a resilient Big Data infrastructure in Singapore. You will lead the engineering of scalable data lakes, data warehouses and streaming pipelines, while ensuring high availability and performance for national security–related analytics.

You will collaborate with data engineers and scientists, manage cluster resources, and drive security, governance and CI/CD integration using Kubernetes and related tools.