Data Engineer (Cloudera)

NCS Pte Ltd; Singapore; On-site; SGD 70,000 - 90,000; Full time

Job summary

A leading technology services firm is seeking a Cloud Deployment & Management Specialist in Singapore. The role involves configuring and managing Cloudera Private Cloud clusters, ensuring high performance, security, and capacity planning. Candidates should have strong Linux and programming skills, along with relevant academic qualifications in Computer Science or related fields.

Qualifications

• Strong Linux command experience and shell scripting.
• Proficiency in Python and SQL.
• Familiarity with DevOps tools like Ansible and GitLab.

Responsibilities

• Install, configure, and maintain Cloudera Private Cloud clusters.
• Manage cluster services like HDFS, Spark, and Kafka.
• Implement security measures and perform performance tuning.

Skills

Linux

Python

SQL

Ansible

TLS/SSL

HDFS

Spark

Kafka

Education

Bachelor’s Degree in Computer Science

Bachelor’s Degree in Information Technology

Bachelor’s Degree in Software Engineering

Diploma in Computer Science

Diploma in Software Engineering

Tools

Cloudera Manager

GitLab

Bamboo

Job description

Company Description

NCS is a leading technology services firm that operates across the Asia Pacific region in over 20 cities, providing consulting, digital services, technology solutions, and more. We believe in harnessing the power of technology to achieve extraordinary things, creating lasting value and impact for our communities, partners, and people. Our diverse workforce of 13,000 has delivered large-scale, mission-critical, and multi-platform projects for governments and enterprises in Singapore and the APAC region.

Job Description

Responsibilities:

1. Cloudera Private Cloud Deployment & Management:

Install, configure, and maintain Cloudera Private Cloud (CDP) clusters.

Manage cluster services (HDFS, YARN, Spark, Hive, Impala, Kafka, etc.).

Monitor cluster health, performance, and capacity planning.

2. Security & Access Control:

Configure and manage Kerberos, LDAP/Active Directory integration, and TLS/SSL encryption. Implement Role-Based Access Control (RBAC) for users and services.

Ensure compliance with security best practices and audit logging.

3. Performance Tuning & Troubleshooting:

Optimize cluster performance by tuning configurations (YARN, HDFS, Spark).

Diagnose and resolve issues related to storage, compute, and networking.

Analyze logs and metrics using Cloudera Manager and other monitoring tools.

4. Data Processing & Optimization:

Develop, optimize, and maintain Spark, Hive, Impala, Nifi, and Kafka applications.

Write efficient SQL queries and optimize Hive/Impala performance.

Build data pipelines for batch and real-time processing

Requirements:

Must-Have Skills:

Linux & Command Line Proficiency: o Strong experience with Linux commands (ls, grep, awk, sed, chmod, ssh).

Managing file permissions, ownership, and shell scripting (Bash).

Programming Languages:

Python (for scripting and automation).

SQL (for database queries and management).

Good-to-Have Skills:

Programming Languages:

Java/Scala (nice to have for application support).

Networking & Security:

Understanding of TLS/SSL, certificates, and data-in-transit encryption.

Knowledge of networking protocols (HTTPS, DNS, NTP) and port management.

Familiarity with Microsoft Active Directory and LDAP authentication.

Scripting & Automation:

Experience with Ansible/Chef/Puppet for infrastructure automation. o Knowledge of GitLab/Bamboo for DevOps and CI/CD pipelines.

Academic Qualifications:

1. Bachelor’s Degree in:

Computer Science / Information Technology

Software Engineering

Data Analytics / Big Data Technologies.

OR

Diploma in:

Computer Science / Information Technology

Software Engineering o Data Analytics / Big Data Technologies

We are driven by ourAEIOU beliefs—Adventure, Excellence, Integrity, Ownership, and Unity—and we seek individuals who embody these values in both their professional and personal lives. We are committed to ourImpact: Valuing our clients, Growing our people, and Creating our future.