Data Engineer/Senior Data engineer-Python, PySpark, SQL, Databricks

Tap Growth ai; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

Tap Growth ai in Singapore is seeking a Data Engineer/Senior Data Engineer with 5+ years of experience to design, build, and maintain robust data pipelines and architectures. You will ensure reliability, scalability, and performance of data infrastructure to support BI and analytics initiatives.

Work from office in Singapore. The role focuses on lakehouse architectures, streaming/batch processing, and end-to-end data engineering workflows using Python, PySpark, Spark, Delta Lake, and AWS

Qualifications

• Strong Python, PySpark and SQL capability, including distributed processing, complex transformations, query optimisation, debugging and automated testing.
• Hands-on experience with Databricks, Apache Spark and Delta Lake for production-scale batch or streaming data engineering.
• Hands-on AWS data engineering experience with a relevant mix of S3, Glue, Lake Formation, IAM, KMS, Lambda, Step Functions, EventBridge, CloudWatch, Redshift and MSK/Kinesis.
• Experience designing lakehouse or data-lake architectures, including layered data patterns, dimensional and domain data modelling and reusable curated datasets.
• Experience implementing incremental ingestion, change data capture, idempotency, late-arriving data handling, backfills, replay and recovery.
• Experience implementing data quality, schema validation, reconciliation, metadata and lineage as automated parts of production pipelines.
• Experience with workflow orchestration, Git-based development, CI/CD, infrastructure as code such as Terraform, and environment promotion controls.
• Ability to implement pipeline observability, operational dashboards, alerting, runbooks, incident diagnosis and service recovery.
• Working knowledge of cloud security, least-privilege access, network boundaries, encryption, secret management, data classification and audit logging.
• Ability to review technical designs and code, establish engineering standards, and guide less-experienced engineers.

Skills

Python

PySpark

SQL

Databricks

Apache Spark

Delta Lake

AWS

S3

Glue

Lake Formation

IAM

KMS

Lambda

Step Functions

EventBridge

CloudWatch

Redshift

Kinesis

Tools

Terraform

Job description

We're Hiring: Data Engineer/Senior Data Engineer!

We are looking for a skilled and driven Data Engineer/Senior Data Engineer to join our team in Singapore. The ideal candidate will have at least 5 years of experience designing, building, and maintaining robust data pipelines and architectures. You will play a key role in ensuring the reliability, scalability, and performance of our data infrastructure to support business intelligence and analytics initiatives.

Location: Singapore, Singapore

Work Mode: Work from Office

Role: Data Engineer/Senior Data Engineer

What You'll Do:B.

Technical & Professional CompetenciesEssential (Must-Have)

• Strong Python, PySpark and SQL capability, including distributed processing, complex transformations, queryoptimisation, debugging and automated testing.
• Hands-on experience with Databricks, Apache Spark and Delta Lake for production-scale batch or streamingdata engineering.
• Hands-on AWS data engineering experience with a relevant mix of S3, Glue, Lake Formation, IAM, KMS, Lambda,Step Functions, Event Bridge, CloudWatch, Redshift and MSK/Kinesis.
• Experience designing lakehouse or data-lake architectures, including layered data patterns, dimensional ordomain data modelling and reusable curated datasets.
• Experience implementing incremental ingestion, change data capture, idempotency, late-arriving data handling,backfills, replay and recovery.
• Experience implementing data quality, schema validation, reconciliation, metadata and lineage as automatedparts of production pipelines.
• Experience with workflow orchestration, Git-based development, CI/CD, infrastructure as code such asTerraform, and environment promotion controls.
• Ability to implement pipeline observability, operational dashboards, alerting, runbooks, incident diagnosis andservice recovery.
• Working knowledge of cloud security, least-privilege access, network boundaries, encryption, secretmanagement, data classification and audit logging.
• Ability to review technical designs and code, establish engineering standards, and guide less-experiencedengineers.

Eager to shape the future of data?