Databricks Lakehouse Engineer: Governance & Performance

Software International Corporation (M) Sdn Bhd

Kuala Lumpur

On-site

MYR 180,000 - 240,000

Full time

3 days ago

Be an early applicant

Application generator

Stand out for this role — generate a tailored resume and cover letter in about a minute.

Get past ATS filters

Job summary

Software International Corporation (M) Sdn Bhd in Malaysia is seeking an experienced Data Engineer to architect and deliver scalable data pipelines on Databricks, with strong governance and security via Unity Catalog. You will lead Dalbricks-based data warehouse and lakehouse implementations, optimize PySpark workloads, and drive DevOps practices across CI/CD and GitHub workflows.

You will collaborate with stakeholders across business and technical teams, mentor junior engineers, and participate

Qualifications

Bachelor's degree in CS/IT/Engineering/Data Science or related field.
5-8 years in Data Engineering, Data Warehousing, or Big Data tech.
Minimum 4+ years hands-on Databricks in enterprise environments.
Experience designing enterprise Data Warehouse solutions and modern Lakehouse architectures.
Strong PySpark development and optimization.
Experience with Unity Catalog, RBAC/ABAC, and data governance.

Responsibilities

Design, develop, and maintain scalable data pipelines and data products using Databricks and PySpark.
Build and optimize ETL/ELT solutions for batch and near real-time processing.
Design and maintain Data Warehouse and Lakehouse solutions for reporting, analytics, and AI/ML use cases.
Develop enterprise data models with Delta Lake and Delta Tables; implement SCDs.
Implement data governance with Unity Catalog, RBAC/ABAC, data lineage and auditing.
Create CI/CD pipelines for Databricks solutions and integrate with GitHub Actions.
Collaborate with stakeholders across business and technical teams; provide mentorship and production support.