Junior Data Engineer | Build & Maintain Data Pipelines

HYPERSCAL SOLUTIONS PTE. LTD.; Singapore; On-site; SGD 60,000 - 90,000; Full time

Job summary

Beyondsoft Consulting (Singapore) Pte. Ltd. seeks a data engineer/backend developer to build and maintain batch and streaming data pipelines. You will map, transform and validate data, support scheduled workflows, and help deploy workloads in containerised environments.

You will troubleshoot pipeline issues, ensure data quality, and collaborate to support CI/CD processes while using Agile practices and SAFe workflows in a Singapore office setting.

Qualifications

• Minimum 2 years of experience in data engineering or backend development.
• Familiarity with Agile development methodologies.
• Working knowledge of SQL and Java or Scala.
• Basic experience with Apache Spark, Apache Flink or Apache Kafka.
• Familiarity with Kubernetes, Docker and S3-compatible object storage.
• Experience with data mapping, transformation, validation and scheduled workflows.
• Familiarity with the SAFe framework.
• Exposure to MinIO or equivalent S3-compatible storage solutions.
• Familiarity with relational databases, Git, CI/CD and automated testing.
• Exposure to Apache Airflow or equivalent workflow orchestration tools.
• Knowledge of batch and stream-processing technologies.

Responsibilities

• Develop and maintain batch and streaming data pipelines.
• Perform data mapping, transformation and validation.
• Support scheduled data workflows and pipeline monitoring.
• Assist in deploying and maintaining data workloads in containerised environments.
• Troubleshoot data pipeline issues and support defect resolution.
• Work with the team to maintain data quality and support CI/CD processes.

Skills

Data engineering

SQL

Java/Scala

Cloud/S3

Kubernetes

Docker

Spark/Flink/Kafka

Airflow

GIT/CI-CD

SAFe

Tools

Kubernetes

Docker

S3-compatible storage

MinIO

Git

CI/CD

Airflow

Spark

Kafka

Job description

Beyondsoft Consulting (Singapore) Pte. Ltd. seeks a data engineer/backend developer to build and maintain batch and streaming data pipelines. You will map, transform and validate data, support scheduled workflows, and help deploy workloads in containerised environments.

You will troubleshoot pipeline issues, ensure data quality, and collaborate to support CI/CD processes while using Agile practices and SAFe workflows in a Singapore office setting.