Databricks Lakehouse Engineer: Governance & Performance

Software International Corporation (M) Sdn Bhd

Kuala Lumpur

In loco

MYR 180.000 - 240.000

Tempo pieno

14 giorni+

Aumenta le possibilità di ricevere una risposta da questo datore di lavoro

Il tuo curriculum e una lettera di presentazione personalizzati, che corrispondono esattamente a ciò che sta cercando.

Supera i filtri ATS

Descrizione del lavoro

Software International Corporation (M) Sdn Bhd in Malaysia is seeking an experienced Data Engineer to architect and deliver scalable data pipelines on Databricks, with strong governance and security via Unity Catalog. You will lead Dalbricks-based data warehouse and lakehouse implementations, optimize PySpark workloads, and drive DevOps practices across CI/CD and GitHub workflows.

You will collaborate with stakeholders across business and technical teams, mentor junior engineers, and participate

Competenze

Bachelor's degree in CS/IT/Engineering/Data Science or related field.
5-8 years in Data Engineering, Data Warehousing, or Big Data tech.
Minimum 4+ years hands-on Databricks in enterprise environments.
Experience designing enterprise Data Warehouse solutions and modern Lakehouse architectures.
Strong PySpark development and optimization.
Experience with Unity Catalog, RBAC/ABAC, and data governance.

Mansioni

Design, develop, and maintain scalable data pipelines and data products using Databricks and PySpark.
Build and optimize ETL/ELT solutions for batch and near real-time processing.
Design and maintain Data Warehouse and Lakehouse solutions for reporting, analytics, and AI/ML use cases.
Develop enterprise data models with Delta Lake and Delta Tables; implement SCDs.
Implement data governance with Unity Catalog, RBAC/ABAC, data lineage and auditing.
Create CI/CD pipelines for Databricks solutions and integrate with GitHub Actions.
Collaborate with stakeholders across business and technical teams; provide mentorship and production support.