Senior Data Engineer: Databricks & Lakehouse Pipelines

Software International Corporation; Kuala Lumpur; Hybrid; MYR 180,000 - 320,000; Full time

Benefits offered by this job

Up to 24 Days Annual Leave

Flexible & Hybrid Working

Performance Bonus

Enhanced EPF Contributions

Training & Development (In-House & Ex-

Travel Allowance & Expense Benefits

Company Trips & Team Events

Long Service Rewards

Job summary

Software International Corporation seeks a senior data engineer to design, develop, and optimize scalable data pipelines and data products using Databricks and PySpark in Kuala Lumpur. The role covers building enterprise Data Warehouses and Lakehouses, governance, and performance tuning for high-volume analytics.

Collaborate with business users and data stewards to deliver robust data platforms, automate deployments, and ensure data quality, security, and scalability across the organization.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.
• 8+ years of experience in Data Engineering, Data Warehousing, or Big Data technologies.
• Minimum 4+ years of hands-on Databricks experience in enterprise environments.
• Experience designing and implementing enterprise Data Warehouse solutions and modern Lakehouse architectures.

Responsibilities

• Design, develop, and maintain scalable data pipelines and data products using Databricks and PySpark.
• Build and optimize ETL/ELT solutions supporting batch and near real-time data processing.
• Design, develop, and maintain enterprise Data Warehouse and Lakehouse solutions for reporting, analytics, and AI/ML use cases.
• Develop and maintain enterprise data models using Data Lake and Delta Tables.
• Implement dimensional data models (Star/Snowflake, SCD) across platforms.
• Ensure data quality, security, and performance across the data platform.
• Implement CI/CD pipelines and DevOps practices for Databricks solutions.
• Engage with stakeholders to translate requirements into scalable technical designs.

Skills

Data engineering

Big data technologies

ETL/ELT design

Stakeholder communication

Education

Bachelor's degree in Computer Science, IT, Engineering, Data Science, or related field

Tools

Databricks

SQL

Python

Delta Lake

Job description

Software International Corporation seeks a senior data engineer to design, develop, and optimize scalable data pipelines and data products using Databricks and PySpark in Kuala Lumpur. The role covers building enterprise Data Warehouses and Lakehouses, governance, and performance tuning for high-volume analytics.

Collaborate with business users and data stewards to deliver robust data platforms, automate deployments, and ensure data quality, security, and scalability across the organization.