Senior Data Engineer: Spark, Kafka, Kubernetes

dci consultants private limited; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

dci consultants private limited is seeking a data engineer in Singapore with 5–8 years of data engineering experience to design and deploy large-scale data systems. The role concentrates on building batch and streaming pipelines and ensuring robust data quality and governance.

You will design pipelines using Spark, Flink and Kafka, program in Java or Scala with SQL, and deploy workloads in Kubernetes. Experience with S3-compatible storage, data-quality controls, schema/lineage, checkpointing,

Qualifications

• 5–8 years of data engineering experience.
• Strong hands-on experience designing batch and streaming pipelines using Apache Spark, Apache Flink and Apache Kafka.
• Proficiency in Java or Scala for distributed data processing, together with SQL.
• Experience deploying and scaling data workloads in containerised Kubernetes environments.
• Experience with S3-compatible object storage, data-quality controls, schema and lineage management, checkpointing and recovery, GitLab and CI/CD practices.

Skills

Data engineering

Spark

Flink

Kafka

Java/Scala

SQL

Kubernetes

S3 storage

Data quality

Schema lineage

Checkpointing

GitLab CI/CD

Job description

dci consultants private limited is seeking a data engineer in Singapore with 5–8 years of data engineering experience to design and deploy large-scale data systems. The role concentrates on building batch and streaming pipelines and ensuring robust data quality and governance.

You will design pipelines using Spark, Flink and Kafka, program in Java or Scala with SQL, and deploy workloads in Kubernetes. Experience with S3-compatible storage, data-quality controls, schema/lineage, checkpointing,