Databricks Data Engineer: Scale Lakehouse Pipelines

K2 Partnering Solutions; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

K2 Partnering Solutions in Singapore is seeking a Databricks Data Engineer to design, build and optimize high-throughput data pipelines on the Databricks Lakehouse platform. You will implement Delta Lake, Unity Catalog and Spark-based processing, collaborate with data science and BI teams, and help enforce CI/CD and testing for reliable data products.

The role emphasizes scalable, secure and cost-aware data infrastructure with hands-on experience in cloud environments (AWS/Azure/GCP) and strong

Qualifications

• Bachelor's degree in Computer Science, Data Engineering, Information Systems, or equivalent practical experience.
• 3+ years of hands-on experience in data engineering, with at least 2+ years actively building on Databricks.
• Strong proficiency in Python (PySpark) and SQL; Scala is a plus.

Responsibilities

• Architecture and deploy robust data pipelines using Apache Spark, PySpark, and Spark SQL to ingest data from diverse sources.
• Implement and manage the Medallion Architecture (Bronze, Silver, Gold) using Delta Lake for clean, analytics-ready datasets.
• Automate batch and streaming workflows using Databricks Lakeflow Jobs, Workflows, or Apache Airflow.
• Set up fine-grained access control, security policies, and data lineage tracking using Unity Catalog.
• Optimize Spark jobs, query performance, and compute cluster configurations to balance performance and cloud costs.
• Partner with Data Scientists, BI Engineers, and product teams to translate requirements into data models and enforce CI/CD testing.

Skills

Data engineering

PySpark

SQL

Python

CI/CD

Git

Spark

Delta Lake

Airflow

Education

Bachelor's degree in Computer Science, Data Engineering, Information Systems, or equivalent practical experience.

Tools

Databricks

Unity Catalog

Delta Lake

Databricks Workflows

Spark SQL

Airflow

dbt

Job description

K2 Partnering Solutions in Singapore is seeking a Databricks Data Engineer to design, build and optimize high-throughput data pipelines on the Databricks Lakehouse platform. You will implement Delta Lake, Unity Catalog and Spark-based processing, collaborate with data science and BI teams, and help enforce CI/CD and testing for reliable data products.

The role emphasizes scalable, secure and cost-aware data infrastructure with hands-on experience in cloud environments (AWS/Azure/GCP) and strong