Senior Data Engineer – Real-Time Pipelines, Spark & Kafka

talentsis pte. ltd.

Singapore

Presencial

SGD 120.000 - 180.000

Jornada completa

Hace 2 días
Sé de los primeros/as/es en solicitar esta vacante

Una candidatura completa en un minuto

Currículum y carta de presentación personalizados, listos para enviar.

Supera los filtros ATS

Descripción de la vacante

Talentsis pte. ltd. in Singapore seeks an experienced Senior Data Engineer to design, develop and maintain large-scale aviation data systems, building batch and real-time pipelines.

You will integrate multiple data sources, storage and recovery mechanisms, deploy on Kubernetes, and ensure data quality and governance across sites.

The role emphasizes GitLab CI/CD, monitoring, and disaster recovery with a focus on reliability and scalability.

Formación

  • 5–8 years of relevant experience in data engineering.
  • Experience designing and developing batch and streaming data pipelines.
  • Proficiency in Java or Scala for distributed data processing and SQL.
  • Experience deploying and scaling data workloads using Docker and Kubernetes.
  • Familiarity with S3-compatible object storage, data schemas, metadata management, data lineage and data quality controls.
  • Experience with checkpointing, fault recovery, data retention and disaster recovery mechanisms.
  • Hands-on experience with GitLab CI/CD pipelines, automated testing and monitoring.
  • Strong analytical, troubleshooting and problem-solving skills.

Responsabilidades

  • Design, develop and maintain batch and real-time data pipelines using Apache Spark, Apache Flink and Apache Kafka.
  • Implement distributed data processing solutions using Java or Scala and SQL.
  • Integrate multiple data sources and S3-compatible object storage for ingestion and processing.
  • Manage data schemas, metadata, lineage and data quality across pipelines.
  • Deploy, maintain and scale data workloads in Kubernetes environments.
  • Implement automated testing, GitLab CI/CD pipelines, monitoring and troubleshooting.
  • Establish checkpointing, recovery, data retention and lifecycle management.
  • Support high availability, backup, disaster recovery and data synchronization across sites.

Conocimientos

Spark
Flink
Kafka
Java
Scala
SQL
Docker
Kubernetes
GitLab CI/CD
Data quality & lineage

Herramientas

Apache Spark
Apache Flink
Apache Kafka
Java
Scala
SQL
Docker
Kubernetes
GitLab CI/CD

Descripción del empleo

Talentsis pte. ltd. in Singapore seeks an experienced Senior Data Engineer to design, develop and maintain large-scale aviation data systems, building batch and real-time pipelines.

You will integrate multiple data sources, storage and recovery mechanisms, deploy on Kubernetes, and ensure data quality and governance across sites.

The role emphasizes GitLab CI/CD, monitoring, and disaster recovery with a focus on reliability and scalability.

Consigue la evaluación confidencial y gratuita de tu currículum.

o arrastra y suelta tu archivo aquí

Similar jobs

Puestos de trabajo similares que vale la pena comparar

Senior Data Systems Engineer - Spark/Kafka on Kubernetes
Senior Data Systems Engineer - Spark/Kafka on Kubernetes

talentsis pte. ltd. • Singapore

Presencial
SGD 120.000 - 180.000
Senior Data Engineer: Real-Time Pipelines & Spark
Senior Data Engineer: Real-Time Pipelines & Spark

hyperscal solutions pte. ltd. • Singapore

Presencial
SGD 90.000 - 140.000
Senior Data Engineer: Real-Time Pipelines on Kubernetes
Senior Data Engineer: Real-Time Pipelines on Kubernetes

Beyondsoft Singapore • Singapore

Presencial
SGD 120.000 - 180.000
Senior Data Engineer: Real-Time Pipelines, K8s & Quality
Senior Data Engineer: Real-Time Pipelines, K8s & Quality

Beyondsoft International (Singapore) Pte Ltd • Singapore

Presencial
SGD 120.000 - 180.000
Senior Data Engineer – Spark, Flink & Kafka on Kubernetes
Senior Data Engineer – Spark, Flink & Kafka on Kubernetes

Singapore Technologies Engineering Ltd • Singapore

Presencial
SGD 120.000 - 180.000
Senior Data Engineer: Real-Time Pipelines & MLOps Architect
Senior Data Engineer: Real-Time Pipelines & MLOps Architect

Singapore Airlines • Singapore

Presencial
SGD 180.000 - 280.000
Senior Data Engineer: Build Scalable Data Pipelines
Senior Data Engineer: Build Scalable Data Pipelines

TALENTSIS PTE. LTD. • Singapore

Presencial
SGD 90.000 - 120.000
Senior Data Engineer (Permanent)
Senior Data Engineer (Permanent)

talentsis pte. ltd. • Singapore

Presencial
SGD 120.000 - 180.000
Senior Data Engineer: Spark, Kafka, Kubernetes
Senior Data Engineer: Spark, Kafka, Kubernetes

dci consultants private limited • Singapore

Presencial
SGD 90.000 - 130.000
Senior Data Engineer: Build Scalable Data Pipelines
Senior Data Engineer: Build Scalable Data Pipelines

TALENTSIS PTE. LTD. • Singapore

Presencial
SGD 70.000 - 110.000