Senior Databricks Data Engineer for AI-Driven Platform

Michael Page; Kuala Lumpur; On-site; MYR 120,000 - 180,000; Full time

Benefits offered by this job

Competitive remuneration package

Benefits package

Enterprise-scale tech exposure

Career development opportunities

Job summary

Michael Page is partnering with a leading multinational technology organisation in Malaysia to hire a Senior Data Engineer. You will design, build, and maintain scalable data pipelines using Databricks and PySpark to support batch and real-time analytics, AI/ML use cases, and enterprise-grade data warehouses.

We seek 8+ years in data engineering, strong Databricks experience, governance and CI/CD expertise, plus stakeholder management.

Qualifications

• Bachelor's Degree in Computer Science, Information Technology, Engineering, Data Science, or a related discipline.
• Minimum 8 years of experience in Data Engineering, Data Warehousing, or Big Data environments.
• At least 4 years of hands-on experience with Databricks in enterprise-scale implementations.
• Strong expertise in PySpark, Data Lake, Delta Tables, ETL/ELT development, and modern Lakehouse architectures.
• Proven experience in designing enterprise Data Warehouse solutions and dimensional data models.
• Solid understanding of data governance, security frameworks, data lineage, and access control models.
• Experience implementing CI/CD pipelines, GitHub integration, and DevOps automation practices.
• Excellent stakeholder management and communication skills with the ability to translate business requirements into technical solutions.
• Strong analytical and troubleshooting capabilities with experience supporting mission-critical data platforms.

Responsibilities

• Design, develop, and maintain scalable data pipelines and data products using Databricks and PySpark.
• Build and optimize ETL/ELT frameworks supporting batch and near real-time processing.
• Develop enterprise Data Warehouse and Lakehouse solutions for analytics, reporting, and AI/ML use cases.
• Create and maintain dimensional data models including Fact and Dimension tables, Star Schema, Snowflake Schema, and Slowly Changing Dimensions (SCD).
• Ensure high standards of data quality, security, scalability, and platform performance.
• Implement Databricks governance solutions including Unity Catalog, RBAC, ABAC, data lineage, auditing, and secure data sharing frameworks.
• Drive platform optimization through advanced PySpark tuning, workload management, cluster optimization, and cost efficiency initiatives.
• Design and implement CI/CD pipelines, deployment automation, and DevOps best practices for Databricks environments.
• Collaborate with business stakeholders, architects, analysts, and technology teams to deliver scalable data solutions.
• Provide technical guidance to team members and participate in production support and incident management activities.

Skills

Databricks

PySpark

ETL/ELT

Data Warehouse

Lakehouse

Dimensional Models

Data Governance

CI/CD

GitHub

DevOps

Education

Bachelor's Degree

Tools

CI/CD pipelines

Job description

Michael Page is partnering with a leading multinational technology organisation in Malaysia to hire a Senior Data Engineer. You will design, build, and maintain scalable data pipelines using Databricks and PySpark to support batch and real-time analytics, AI/ML use cases, and enterprise-grade data warehouses.

We seek 8+ years in data engineering, strong Databricks experience, governance and CI/CD expertise, plus stakeholder management.