Data Engineer / Senior Data Engineer

RAPSYS TECHNOLOGIES PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

RAPSYS TECHNOLOGIES PTE. LTD. is seeking a data engineering professional with strong Python, PySpark and SQL skills, plus hands-on experience with Databricks, Spark and Delta Lake for production-scale pipelines in a cloud environment.

The candidate will design lakehouse architectures, implement incremental ingestion and ensure data quality, security and observability across data platforms. Required experience covers AWS data services (S3, Glue, Lake Formation, IAM, KMS, Lambda, Step Functions,

Qualifications

• Strong Python, PySpark and SQL capabilities with distributed processing and testing.
• Production-scale experience with Databricks, Spark and Delta Lake.
• Hands-on AWS data engineering with S3, Glue, Lake Formation, IAM, KMS, Lambda and related services.
• Experience designing lakehouse architectures and reusable data patterns.
• Experience implementing incremental ingestion, CDC, idempotency and backfills.

Skills

Python

PySpark

SQL

Databricks

Spark

Delta Lake

AWS

S3

Glue

Lake Formation

IAM

KMS

Lambda

Step Functions

EventBridge

CloudWatch

Redshift

Kinesis

Data Lake Architecture

Incremental Ingestion

CDC

Backfills

Data Quality

Schema Validation

Metadata & Lineage

CI/CD

Terraform

Infrastructure as Code

Observability

Dashboards

Security

Encryption

Secret Management

Data Classification

Audit Logging

Review Designs

Tools

Databricks Unity Catalog

Lakeflow

Delta Live Tables

Auto Loader

Structured Streaming

Delta Sharing

Databricks Asset Bundles

Apache Airflow

dbt

Kafka

AWS DMS

MSK

Kinesis

Job description

Essential (Must-Have)

• Strong Python, PySpark and SQL capability, including distributed processing, complex transformations, query optimisation, debugging and automated testing.
• Hands-on experience with Databricks, Apache Spark and Delta Lake for production-scale batch or streaming data engineering.
• Hands-on AWS data engineering experience with a relevant mix of S3, Glue, Lake Formation, IAM, KMS, Lambda, Step Functions, Event Bridge, CloudWatch, Redshift and MSK/Kinesis.
• Experience designing lakehouse or data-lake architectures, including layered data patterns, dimensional or domain data modelling and reusable curated datasets.
• Experience implementing incremental ingestion, change data capture, idempotency, late-arriving data handling, backfills, replay and recovery.
• Experience implementing data quality, schema validation, reconciliation, metadata and lineage as automated parts of production pipelines.
• Experience with workflow orchestration, Git-based development, CI/CD, infrastructure as code such as Terraform, and environment promotion controls.
• Ability to implement pipeline observability, operational dashboards, alerting, runbooks, incident diagnosis and service recovery.
• Working knowledge of cloud security, least-privilege access, network boundaries, encryption, secret management, data classification and audit logging.
• Ability to review technical designs and code, establish engineering standards, and guide less-experienced engineers.

Preferred (Advantageous)

• Experience with Databricks Unity Catalog, Lakeflow or Delta Live Tables, Auto Loader, Structured Streaming, Delta Sharing and Databricks Asset Bundles.
• Experience with Apache Airflow, dbt, Kafka, Amazon MSK/Kinesis, AWS DMS or equivalent ingestion and orchestration technologies.
• Experience designing data products, data contracts, semantic or context layers, master/reference data patterns or data mesh-aligned operating models.
• Experience supporting feature engineering, ML training datasets, feature stores or MLOps pipelines on Databricks and AWS.
• Experience in Singapore public-sector environments, including IM8-aligned controls, data classification, VAPT, system assurance or governed cross-agency data sharing.
• Relevant certifications such as Databricks Certified Data Engineer Professional, AWS Certified Data Engineer –Associate or AWS Certified Solutions Architect – Associate/Professional.