Senior Data Engineer - Databricks, Azure & Lakehouse

Unison Consulting Pte Ltd; Kuala Lumpur; On-site; MYR 90,000 - 120,000; Full time

Job summary

A consulting firm in Kuala Lumpur is seeking a Senior Data Engineer with expertise in Databricks, PySpark, and Delta Lake. The candidate will design scalable ETL/ELT solutions, implement data architectures, and lead projects involving data ingestion from various sources. The ideal applicant should have 5–8+ years of Data Engineering experience along with strong technical leadership skills. This role includes automation and optimization of workflows, offering a chance to guide junior team members in a dynamic environment.

Qualifications

• 5–8+ years of experience in Data Engineering.
• Strong hands‑on experience with Databricks, PySpark, Delta Lake, SQL, Python.
• Experience with Azure Data Lake, ADF, or AWS equivalents.
• Ability to optimize Spark workloads.
• Familiarity with Git, Azure DevOps CI/CD, and YAML pipelines.

Responsibilities

• Design, build, and optimize data ingestion and transformation pipelines.
• Implement Delta Lake and Medallion architecture.
• Develop ingestion frameworks for data sources.
• Automate workflows using Databricks Workflows and Azure Functions.
• Provide technical leadership and guidance to junior team members.

Skills

Data Engineering

Databricks

PySpark

Delta Lake

SQL

Python

Azure Data Lake

ADF

Azure Functions

Git

Tools

Terraform

Power BI

PostgreSQL

Job description

A consulting firm in Kuala Lumpur is seeking a Senior Data Engineer with expertise in Databricks, PySpark, and Delta Lake. The candidate will design scalable ETL/ELT solutions, implement data architectures, and lead projects involving data ingestion from various sources. The ideal applicant should have 5–8+ years of Data Engineering experience along with strong technical leadership skills. This role includes automation and optimization of workflows, offering a chance to guide junior team members in a dynamic environment.