Senior Data Engineer [20261042355]

Beyondsoft International (Singapore) Pte Ltd; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Beyondsoft International (Singapore) Pte. Ltd. invites a senior Data Engineer to design, develop and deploy large-scale batch and real-time data processing pipelines.

You will build distributed solutions, manage data storage, ensure data quality and lineage, and scale workloads in Kubernetes environments. You will implement recovery strategies, maintain CI/CD pipelines, and support automated deployments using tools like Apache Airflow, Spark, Flink and Kafka.

Qualifications

• 5+ years of data engineering experience.
• Hands-on experience with Apache Spark, Apache Flink and Apache Kafka.
• Proficiency in Java or Scala and SQL.
• Experience with Kubernetes and S3-compatible object storage.
• Knowledge of data quality controls, schema management, data lineage, checkpointing and recovery.
• Familiarity with GitLab and CI/CD practices.
• Experience with MinIO or equivalent S3-compatible storage solutions.
• Knowledge of Bronze/Silver/Gold data architecture and data integrity controls.
• Experience in high availability, backup, disaster recovery and inter-site synchronisation.
• Experience with Apache Airflow or equivalent workflow orchestration tools.
• Familiarity with SAFe and safety-critical or mission-critical systems.

Responsibilities

• Design, develop and deploy large-scale batch and real-time data processing pipelines.
• Build and optimise distributed data processing solutions.
• Deploy, manage and scale data workloads in Kubernetes environments.
• Manage data storage, quality, schema, metadata and lineage.
• Implement data checkpointing, recovery, backup and disaster recovery solutions.
• Ensure data integrity, retention and high availability across distributed storage systems.
• Maintain CI/CD pipelines and support automated deployment processes.

Skills

Spark

Flink

Kafka

Java

Scala

SQL

Airflow

Data quality

Data lineage

CI/CD

SAFe

Safety-critical

Tools

Kubernetes

MinIO

GitLab

S3-compatible storage

Job description

COMPANY DESCRIPTION

Beyondsoft International (Singapore) Pte. Ltd. was set up in 2007 and established as the regional headquarters for the Southeast Asia (SEA) and European markets in September 2015. Based on our vision of "Using technology to promote social progress, economic development and become a global customer preferred partner" and our concept of "Beyond your expectations", Beyondsoft is committed to provide our customers in countries along the "Belt and Road" with comprehensive solutions and products and creating commercial value for customers to realizing continuous businesses development.

Our core business includes:

1. IT development services providing customers with IT consulting, software research and development, software and hardware testing, system integration and operation and maintenance, data analysis and other services;
2. New retail solutions and products through intelligent products, helping small and medium-sized enterprises (SMEs) realize the digital transformation of their daily operations;
3. Internet of Things (IoT) platform and solutions comprehensive use of IoT, artificial intelligence, big data, cloud computing and other technologies to provide IoT solutions for intelligent upgrades in cities, parks, buildings and industries, to create a smart future.

For more information, please visit www.beyondsoft.com.

RESPONSIBILITIES

• Design, develop and deploy large-scale batch and real-time data processing pipelines.

• Build and optimise distributed data processing solutions.

• Deploy, manage and scale data workloads in Kubernetes environments.

• Manage data storage, quality, schema, metadata and lineage.

• Implement data checkpointing, recovery, backup and disaster recovery solutions.

• Ensure data integrity, retention and high availability across distributed storage systems.

• Maintain CI/CD pipelines and support automated deployment processes.

QUALIFICATIONS

• Minimum 5 years of relevant data engineering experience.

• Hands-on experience with Apache Spark, Apache Flink and Apache Kafka.

• Proficiency in Java or Scala and SQL.

• Experience with Kubernetes and S3-compatible object storage.

• Knowledge of data quality controls, schema management, data lineage, checkpointing and recovery.

• Familiarity with GitLab and CI/CD practices.

• Experience with MinIO or equivalent S3-compatible storage solutions.

• Knowledge of Bronze/Silver/Gold data architecture and data integrity controls.

• Experience in high availability, backup, disaster recovery and inter-site synchronisation.

• Experience with Apache Airflow or equivalent workflow orchestration tools.

• Familiarity with SAFe and safety-critical or mission-critical systems.