Data Engineer: Spark, Scala & Elasticsearch on OpenShift

Hong Kong Unison Limited; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Unison Consulting in Singapore is seeking a talented Data Engineer with deep expertise in Hadoop, Spark and Elasticsearch to design and optimize big data solutions. You will build scalable data processing pipelines using Spark and Scala and integrate Elasticsearch for powerful search capabilities, while collaborating with cross-functional teams in a regulated domain.

Candidates should bring hands-on experience with OpenShift Container Platform (OCP), Docker, and DevOps practices to ensure

Qualifications

• Proven experience as a Data Engineer in large-scale environments.
• Experience with Hadoop, Spark, and data processing technologies.
• Proficiency in Scala and Spark SQL.
• Experience with Elasticsearch integration and data modeling.
• Knowledge of DevOps practices and containerization (OpenShift, Kubernetes).

Responsibilities

• Implement data transformation, aggregation, and enrichment pipelines.
• Design Spark Scala applications and data processing workflows.
• Integrate Elasticsearch with Spark for indexing and search.
• Tune Spark jobs for performance and scalability.
• Monitor job performance and data quality across pipelines.

Skills

Hadoop

Apache Spark

Scala

Elasticsearch

DevOps

OpenShift

Kubernetes

Quantexa

Data pipelines

CI/CD

Job description

Unison Consulting in Singapore is seeking a talented Data Engineer with deep expertise in Hadoop, Spark and Elasticsearch to design and optimize big data solutions. You will build scalable data processing pipelines using Spark and Scala and integrate Elasticsearch for powerful search capabilities, while collaborating with cross-functional teams in a regulated domain.

Candidates should bring hands-on experience with OpenShift Container Platform (OCP), Docker, and DevOps practices to ensure