Data Engineer: Spark-Scala, Elasticsearch & OCP Expert

Synechron Technologies Singapore PTE Ltd; Singapore; On-site; SGD 90,000 - 180,000; Full time

Job summary

Synechron Technologies Singapore PTE Ltd is seeking a skilled Data Engineer to design, develop, and optimize big data solutions using Hadoop, Spark, Scala, and Elasticsearch. You will work with cross-functional teams to build scalable data pipelines and search applications, with Quantexa experience highly preferred.

The ideal candidate will have strong data engineering experience, proficiency in Spark/Scala, and hands-on work with OpenShift Container Platform (OCP) and DevOps practices to ensure

Qualifications

• Must be Quantexa certified data engineer / data architect.
• Experience with Hadoop, Spark, and data processing in large-scale environments.
• Proficiency in Scala and familiarity with functional programming concepts.
• Experience with Quantexa software is highly preferred.
• Knowledge of OpenShift Container Platform (OCP) and container orchestration.

Responsibilities

• Implement data transformation, aggregation, and enrichment processes for data analytics and ML initiatives.
• Collaborate with cross-functional teams to translate data requirements into engineering solutions.
• Design and implement Spark Scala applications and data pipelines for large structured and unstructured data.
• Integrate Elasticsearch with Spark for indexing, searching, and data retrieval.
• Tune Spark jobs for performance and scalability; optimize Elasticsearch indexing and queries.
• Develop and maintain scalable, fault-tolerant Spark applications following best practices.
• Troubleshoot data processing, performance, and data quality issues in Spark-Elasticsearch integration.
• Monitor job performance metrics and propose optimizations for Spark and Elasticsearch components.
• Ensure data quality and integrity throughout the data pipeline lifecycle.
• Design and deploy data engineering solutions on OpenShift Container Platform (OCP).
• Optimize data workflows for containerized deployment and resource usage.
• Collaborate with DevOps to implement CI/CD and platform stability.

Skills

Scala programming

Spark

Hadoop

Data processing

Data modeling

Big data

Education

Bachelor's or Master's degree in Computer Science, Software Engineering, or related field

Tools

OpenShift Container Platform (OCP)

Elasticsearch

Apache Spark

Kubernetes

Docker

Jenkins

Git

Job description

Synechron Technologies Singapore PTE Ltd is seeking a skilled Data Engineer to design, develop, and optimize big data solutions using Hadoop, Spark, Scala, and Elasticsearch. You will work with cross-functional teams to build scalable data pipelines and search applications, with Quantexa experience highly preferred.

The ideal candidate will have strong data engineering experience, proficiency in Spark/Scala, and hands-on work with OpenShift Container Platform (OCP) and DevOps practices to ensure