Senior Data Engineer: Lakehouse Architect & Pipelines

JOBSTER PRIVATE LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

JOBSTER PRIVATE LTD. seeks a seasoned Data Engineer to design and deliver maintainable batch and streaming data pipelines on AWS and Databricks.

You will build bronze-to-gold data models, enforce data quality and governance, and optimise performance and cost across large-scale datasets. You will lead data integration from databases, files and APIs, implement CD/CI and IaC, and craft reliable monitoring, dashboards and runbooks to ensure data freshness and downstream reliability for business

Qualifications

• Degree in Computer Science, Software Engineering, Data Engineering, Information Systems or a related technical discipline.

Responsibilities

• Lead the design and implementation of maintainable batch and streaming pipelines in AWS and Databricks.
• Design Bronze, Silver and Gold data layers using Delta Lake patterns.
• Implement reliable ingestion from databases, files, APIs and event streams.
• Establish schema enforcement, data contracts and automated validation.

Skills

Python

PySpark

SQL

Delta Lake

Databricks

Spark

AWS

CI/CD

Terraform

Observability

Security basics

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

Apache Spark

Delta Lake

Terraform

Git

Job description

JOBSTER PRIVATE LTD. seeks a seasoned Data Engineer to design and deliver maintainable batch and streaming data pipelines on AWS and Databricks.

You will build bronze-to-gold data models, enforce data quality and governance, and optimise performance and cost across large-scale datasets. You will lead data integration from databases, files and APIs, implement CD/CI and IaC, and craft reliable monitoring, dashboards and runbooks to ensure data freshness and downstream reliability for business