Senior Data Platform System Engineer (Data Engineering Team) (Data & AI)

SPGroup; Singapore; On-site; SGD 120,000 - 180,000; Full time

Benefits offered by this job

Wellness programmes

Job summary

SP Group is seeking an experienced Data Platform Engineer to operate and evolve our enterprise Data & AI platform, supporting analytics, regulatory compliance, and AI-driven innovation. The role focuses on scalability, reliability, security, and performance to enable data engineers, scientists, and analysts to deliver business value efficiently.

You will design, deploy, and manage large-scale data platform environments, optimize Hadoop ecosystem components, and ensure platform uptime with

Qualifications

• 5–10 years of experience in data platform engineering, big data operations, or infrastructure engineering.
• Strong hands-on experience managing big data platforms (Cloudera CDP, Hadoop ecosystem).
• Deep understanding of Hadoop components: HDFS, Hive, Spark, HBase, and related services.
• Proficiency in automation and scripting (Python, Bash, Terraform, Ansible).
• Experience with CI/CD pipelines and DevOps practices.
• Solid understanding of data platform security and access control concepts.
• Strong analytical and problem-solving skills with a proactive mindset.

Responsibilities

• Ensure high availability and reliability of the enterprise data platform (Cloudera and Talend Big Data).
• Deliver scalable and performant data infrastructure supporting batch and near real-time workloads.
• Enable secure and compliant data operations aligned with regulatory requirements.
• Improve platform automation and operational efficiency through DevOps and IaC.
• Provide platform support to data engineering, analytics, and business teams.
• Plan and coordinate platform upgrades, patching, and scaling activities.

Skills

Strong analytical skills

Communication skills

Ownership mentality

Cross-functional collaboration

Tools

Cloudera CDP

Hadoop ecosystem

HDFS

Hive

Spark

HBase

Iceberg

Ozone

Terraform

Ansible

CI/CD

Python

Bash

Job description

You will be responsible for operating and evolving SP Group’s enterprise Data & AI platform

to support mission-critical analytics, regulatory compliance, and AI-driven innovation. This

role focuses on ensuring the platform is scalable, reliable, secure, and performant, while

enabling data engineers, scientist and analysts to deliver business value efficiently.

What You'll Do

• Ensure high availability and reliability of the enterprise data platform (Cloudera and Talend Big Data).
• Deliver scalable and performant data infrastructure supporting batch and near real-time workloads.
• Enable secure and compliant data operations aligned with enterprise and regulatory requirements.
• Improve platform automation and operational efficiency through DevOps and Infrastructure-as-Code.
• Provide strong platform support and partnership to data engineering, analytics, and business teams.

Data Platform Engineering & Operations

• Design, deploy, and manage large-scale data platform environments (primarily Cloudera and Talend Big Data).
• Operate and optimize Hadoop ecosystem components (HDFS, Hive, Spark, HBase, Impala, Iceberg, Ozone).
• Manage cluster lifecycle including provisioning, scaling, patching, and upgrades.
• Ensure platform uptime, stability, and service reliability through proactive monitoring and maintenance.

Performance, Scalability & Capacity Management

• Monitor workload performance and optimize data processing efficiency.
• Perform capacity planning to support increasing data volumes and usage.
• Troubleshoot performance bottlenecks across compute, storage, and query layers.

Platform Delivery & Execution Planning

• Plan and coordinate execution of platform upgrades, patching, and scaling activities.
• Manage deployment schedules and ensure minimal service disruption.
• Track and report execution progress against platform roadmap.
• Implement Infrastructure-as-Code using tools such as Terraform and Ansible.
• Design and maintain CI/CD pipelines for platform configuration.
• Automate platform provisioning, deployment, and operational workflows.
• Improve observability through monitoring, alerting, and logging practices.

Security, Governance & Compliance

• Implement and manage platform security controls including authentication (Kerberos), authorization (e.g. Ranger), and encryption.
• Ensure compliance with enterprise data governance, security policies, and regulatory requirements.
• Support auditability, access control, and data protection mechanisms.
• Work closely with data engineers, data scientists, and business teams to enable data solution delivery.
• Partner with infrastructure, network, and security teams to ensure platform integration and stability.
• Provide guidance, troubleshooting, and best practices to users of the platform.
• Document platform architecture, processes, and operational procedures.

What You'll Need

• 5- 10 years of experience in data platform engineering, big data operations, or infrastructure engineering.
• Strong hands-on experience managing big data platforms (Cloudera CDP, Hadoop ecosystem).
• Deep understanding of Hadoop components: HDFS, Hive, Spark, HBase, and related services.
• Experience with cluster operations, performance tuning, and troubleshooting.
• Proficiency in automation and scripting (Python, Bash, Terraform, Ansible).
• Experience with CI/CD pipelines and DevOps practices.
• Solid understanding of data platform security and access control concepts.
• Strong analytical and problem-solving skills with a proactive mindset.
• Ability to work in a fast-paced, cross-functional environment.
• Good communication skills with ability to translate technical topics to stakeholders.
• Ownership mentality with focus on platform reliability and continuous improvement.

Good to have

• Experience with Databricks, Talend, or hybrid data platforms.
• Knowledge of containerization and orchestration.
• Experience with streaming platforms (e.g. Kafka) or real-time processing.
• Familiarity with modern data lake-house concepts (e.g. Iceberg, Delta).
• Exposure to cloud or hybrid environments (Azure preferred).

What We'll Provide

• Join us to empower the future of energy .
• You will work in teams that thrive on collaboration for outstanding achievements.
• We have regular wellness programmes to promote healthy living