Senior Data Engineer

exceltech computers pte. ltd.; Singapore; In loco; SGD 120.000 - 180.000; Tempo pieno

Descrizione del lavoro

Exceltech Computers Pte. Ltd. is seeking a Senior Data Engineer based at Paya Lebar to design, build and operate trusted data pipelines and reusable data products on our data platform using AWS and Databricks.

The role requires 6–10 years of data or software engineering experience, with hands-on production data pipelines and at least 2 years on cloud-based data platforms. You will work within the DSAD Platform and Architecture Team to drive scalable data solutions.

Competenze

• 6–10 years of relevant data or software engineering experience.
• Hands-on experience with production data pipelines.
• At least 2 years working with cloud-based data platforms.
• Strong Python, PySpark and SQL capability.
• Databricks, Apache Spark and Delta Lake in production-scale data engineering.
• Experience with AWS data engineering including S3, Glue, Lake Formation, IAM, KMS, Lambda, Step Functions, EventBridge, CloudWatch, Redshift and MSK/Kinesis.
• Experience designing lakehouse/data-lake architectures with dimensional or domain data modelling.
• Experience implementing data quality, metadata and lineage in production pipelines.

Mansioni

• Design, build, productionise and operate trusted data pipelines and reusable data products on the data platform using AWS and Databricks.

Conoscenze

Python

PySpark

SQL

Strumenti

Databricks

Apache Spark

Delta Lake

AWS S3

AWS Glue

Lake Formation

IAM

KMS

Lambda

Step Functions

EventBridge

CloudWatch

Redshift

MSK/Kinesis

Descrizione del lavoro

Senior Data Engineer
Work location- Paya Lebar

The Senior Data Engineer sits within the DSAD Platform and Architecture Team and is responsible for designing,

building, productionising and operating trusted data pipelines and reusable data products on the data platform using AWS and Databricks.

Experience: Typically 6–10 years of relevant data or software engineering experience, including hands-on design and

operation of production data pipelines and at least 2 years working with cloud-based data platforms.

• Strong Python, PySpark and SQL capability, including distributed processing, complex transformations, query optimisation, debugging and automated testing.


• Hands-on experience with Databricks, Apache Spark and Delta Lake for production-scale batch or streaming data engineering.


• Hands-on AWS data engineering experience with a relevant mix of S3, Glue, Lake Formation, IAM, KMS, Lambda, Step Functions, EventBridge, CloudWatch, Redshift and MSK/Kinesis.


• Experience designing lakehouse or data-lake architectures, including layered data patterns, dimensional or domain data modelling and reusable curated datasets.


• Experience implementing incremental ingestion, change data capture, idempotency, late-arriving data handling, backfills, replay and recovery.


• Experience implementing data quality, schema validation, reconciliation, metadata and lineage as automated parts of production pipelines.


• Experience with workflow orchestration, Git-based development, CI/CD, infrastructure as code such as Terraform, and environment promotion controls.