Quantexa-Certified Lead Data Engineer — Spark & OpenShift

D L RESOURCES PTE LTD

Singapore

On-site

SGD 180,000 - 260,000

Full time

14 days+

Application generator

Don’t send a generic resume — generate a resume and cover letter tailored to this exact role.

Get past ATS filters

Job summary

D L RESOURCES PTE LTD in Singapore seeks a Lead Data Engineer to design, build, and optimize our data infrastructure to support advanced analytics for the bank. You will lead a team of data engineers and collaborate with analysts, DevOps, and infrastructure teams to deliver scalable data solutions using Spark, Elasticsearch, and OpenShift.

The role requires a Quantexa certification, 10+ years in data engineering, and strong skills in Spark, Scala, Python, and Kubernetes.

Qualifications

Bachelor's degree in Computer Science, Data Engineering, Information Technology, or related field.
At least 10 years of experience as a Data Engineer in large-scale environments with Hadoop, Spark, and data processing.
Must be Quantexa certified data engineer / data architect and proficient with the tool.
Strong expertise in designing and developing data infrastructure using Hadoop, Spark, and related tools (HDFS, Hive, Ranger, etc)
Experience with OpenShift Container Platform (OCP) and Kubernetes.
Proficiency in Spark, Python, Scala, or Java.
Knowledge of DevOps, CI/CD pipelines, and automation tools (Docker, Jenkins, Ansible, Bitbucket).
Experience with Grafana, Prometheus, Splunk is a plus.
Excellent collaboration and communication skills; able to lead and mentor.

Responsibilities

Design, develop, and implement Spark Scala applications and data processing pipelines for large data volumes.
Integrate Elasticsearch with Spark for efficient indexing and retrieval.
Tune Spark jobs for performance and scale; ensure data processing efficiency.
Collaborate with engineers, data scientists, and stakeholders to translate requirements into solutions.
Design and deploy data engineering solutions on OpenShift Container Platform using containers.
Optimize workflows for containerized deployment and resource utilization.
Work with DevOps to streamline deployments, CI/CD, and platform stability.
Monitor data pipelines, troubleshoot, and implement enhancements.
Implement monitoring and logging for data infra health and performance.
Document processes and configurations for knowledge sharing.
Stay updated on emerging data engineering and DevOps practices.
Provide technical leadership, mentorship, and continuous improvement of analytics capabilities.