Data Engineer

PageGroup Corporation Solutions (Malaysia) Sdn Bhd; Singapore; On-site; SGD 80,000 - 120,000; Full time

Job summary

PageGroup Corporation Solutions (Malaysia) Sdn Bhd is seeking a Data Engineer with 4-7+ years of experience to design and maintain scalable data pipelines. The role focuses on data ingestion from APIs and enterprise systems, data reliability, and supporting enterprise data platforms.

The engineer will drive data quality, implement CDC and incremental loads, and collaborate with stakeholders on data contracts while contributing to governance, security, and runbooks.

Qualifications

• 4-7+ years of experience in Data Engineering.
• Strong programming skills in Python and SQL.
• Experience building data pipelines from APIs and relational databases.
• Hands-on experience with CDC, incremental loading, data migration and historical backfills.
• Strong experience with Apache Spark.

Responsibilities

• Design and develop data pipelines to ingest data from APIs, databases, and enterprise systems.
• Build and maintain data ingestion frameworks for centralized data platforms.
• Develop solutions for data recovery, replay, and pipeline resilience.
• Ensure data quality through deduplication, incremental loading, and handling late-arriving data.
• Create dashboards and monitoring solutions for data pipelines.
• Implement automated alerts and remediation processes.
• Troubleshoot production issues and participate in on-call rotations.
• Define data requirements and data contracts with stakeholders.
• Maintain technical documentation and runbooks.

Skills

Python

SQL

Data Engineering

Linux

Git

APIs

Tools

Apache Spark

Airflow

Dagster

Prefect

Kafka Connect

Debezium

Iceberg

Delta Lake

Hudi

Kubernetes

Job description

Data Engineer

Job Summary

We are looking for a Data Engineer with 4-7+ years of experience in building and maintaining scalable data pipelines. You will be responsible for designing data ingestion solutions, managing historical data loads, ensuring data reliability, and supporting enterprise data platforms.

Key Responsibilities

Data Engineering

• Design and develop data pipelines to ingest data from APIs, databases, and enterprise systems.
• Build and maintain data ingestion frameworks for centralized data platforms.
• Develop solutions for data recovery, replay, and pipeline resilience.
• Ensure data quality through deduplication, incremental loading, and handling late-arriving data.

Data Migration & Backfill

• Perform historical data migrations and large-scale backfills.
• Validate migrated data and reconcile it with source systems.
• Support historical data loading for newly onboarded business units or sites.

Monitoring & Support

• Create dashboards and monitoring solutions for data pipelines.
• Implement automated alerts and remediation processes.
• Troubleshoot production issues and participate in on‑call support rotations.

Governance & Collaboration

• Work with stakeholders to define data requirements and data contracts.
• Ensure compliance with data governance, security, and retention policies.
• Maintain technical documentation and operational runbooks.
• Participate in code reviews and contribute to engineering best practices.

Required Skills & Experience

Technical Skills

• 4-7+ years of experience in Data Engineering.
• Strong programming skills in Python and SQL.
• Experience building data pipelines from APIs and relational databases.
• Hands‑on experience with:
• Change Data Capture (CDC)
• Incremental data loading
• Data migration and historical backfills
• Strong experience with Apache Spark.
• Experience with modern data lake technologies such as:
• Apache Iceberg
• Delta Lake
• Apache Hudi
• Experience with workflow orchestration tools:
• Apache Airflow
• Dagster
• Prefect
• Strong knowledge of:
• Linux
• Docker
• Git
• Experience supporting production environments and resolving incidents.

Preferred Experience

• Experience managing on‑premise or self‑hosted data platforms.
• Understanding of data governance, security, and access controls.
• Experience implementing data contracts with upstream systems.
• Exposure to large‑scale time‑series or telemetry data.
• Familiarity with:
• Airbyte
• Debezium
• Kafka Connect
• dlt

Nice-to-Have Skills

• Kubernetes
• Trino or other distributed SQL engines
• S3-compatible object storage (MinIO, Ceph, Dell ObjectScale)
• Experience integrating Finance, HR, Procurement, or ERP systems

Contact: Lily Tran (Lic No: R24123492/ EA no:18S9099)

Phone number: [phone]