Databricks Data Engineer — Real-time Pipelines & Delta Lake

ECLATPRIME DIGITAL PTE. LTD.; Singapore; Vor Ort; SGD 90.000 - 130.000; Vollzeit

Zusammenfassung

ECLATPRIME DIGITAL PTE. LTD. in Singapore is seeking a Data Engineer with 5+ years of data engineering and Databricks production experience. You will design scalable Databricks data pipelines, build batch and real-time ingestion workflows, and optimize Spark jobs for performance and cost efficiency.

Responsibilities include implementing Unity Catalog data governance and developing CI/CD pipelines to support data and ML workflows. Immediate to 30 days notice preferred; Singapore PR required.

Qualifikationen

• 5+ years of data engineering experience.
• Hands-on Databricks production experience.
• Singapore PR eligibility.
• Immediate to 30 days notice period.

Aufgaben

• Design and develop scalable Databricks data pipelines.
• Build batch and real-time ingestion workflows.
• Optimize Spark jobs for performance and cost efficiency.
• Implement data security, governance and lineage using Unity Catalog.
• Develop CI/CD pipelines and support data/ML workflows with MLflow.

Kenntnisse

Databricks

PySpark

Python

SQL

Medallion Architecture

Delta Lake

Lakeflow

Auto Loader

Structured Streaming

CDC / SCD Type 1 & 2

Unity Catalog

Data Governance

Lineage

Databricks Workflows

Serverless Compute

Photon

Liquid Clustering

Z-Ordering

Git

Terraform

CI/CD

Tools

Git

Terraform

CI/CD

Jobbeschreibung

ECLATPRIME DIGITAL PTE. LTD. in Singapore is seeking a Data Engineer with 5+ years of data engineering and Databricks production experience. You will design scalable Databricks data pipelines, build batch and real-time ingestion workflows, and optimize Spark jobs for performance and cost efficiency.

Responsibilities include implementing Unity Catalog data governance and developing CI/CD pipelines to support data and ML workflows. Immediate to 30 days notice preferred; Singapore PR required.