Lead Data Engineer

E-Outsource Asia; Kuala Lumpur; On-site; MYR 180,000 - 300,000; Full time

Job summary

E-Outsource Asia in Malaysia seeks a Senior Data Engineer to design and evolve scalable cloud data architectures, data lakes, and data warehouses. You will lead batch and streaming ETL/ELT pipelines, handling high-volume, multi-source data, and ensure data integrity across systems.

Collaboration with Data Architects, Analysts, and Solution Architects will drive governance, security, and compliant data solutions while enabling analytics, dashboards, and AI/ML initiatives with trusted datasets.

Qualifications

• Bachelor's Degree in Computer Science, Information Technology, Software Engineering, Data Science, or related discipline.
• Minimum 8-12 years of experience in data engineering, data integration, or related fields.
• Strong understanding of data warehousing, data lakes, and modern data architecture principles.
• Experience with ETL/ELT tools and data integration platforms.
• Proficiency in SQL and experience with Python, Scala, or similar programming languages.
• Hands-on experience with cloud data platforms such as AWS, Azure, or Google Cloud.
• Experience integrating enterprise applications such as SAP, Salesforce, CRM, ERP, or similar business systems.
• Knowledge of data modeling concepts, including dimensional and relational modeling.
• Familiarity with data governance, data quality, and information security practices.
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and stakeholder management capabilities.

Responsibilities

• Design, evolve, and maintain scalable cloud data architectures, data lakes, and enterprise data warehouses.
• Build and support data integration between enterprise systems, including SAP, Salesforce, CRM platforms, and cloud-based applications.
• Lead the development of batch and streaming ETL/ELT pipelines processing high-volume, multi-source data.
• Implement framework solutions for data monitoring, lineage, data masking, encryption, and role-based access controls
• Implement data models and transformation logic in alignment with enterprise data architecture standards.
• Ensure data integrity, accuracy, and consistency across source and target systems.
• Collaborate with Data Architects, Business Analysts, and Solution Architects in designing scalable data solutions.
• Support analytics, reporting, dashboards, and AI/ML initiatives by providing trusted and governed datasets.
• Monitor and troubleshoot data pipelines, integration jobs, and platform performance issues.
• Implement data governance, security, privacy, and compliance requirements within data solutions.
• Maintain technical documentation, data dictionaries, and data lineage artifacts.
• Identify opportunities for automation and continuous improvement in data engineering processes.

Skills

Data warehousing

Data lakes

ETL/ELT

SQL

Python

Scala

Cloud data platforms

Data governance

Data modeling

Communication

Stakeholder management

Education

Bachelor's degree in CS/IT/SE/Data Science or related

Tools

SAP

Salesforce

CRM platforms

AWS

Azure

GCP

Job description

• Design, evolve, and maintain scalable cloud data architectures, data lakes, and enterprise data warehouses.
• Build and support data integration between enterprise systems, including SAP, Salesforce, CRM platforms, and cloud-based applications.
• Lead the development of batch and streaming ETL/ELT pipelines processing high-volume, multi-source data.
• Implement framework solutions for data monitoring, lineage, data masking, encryption, and role-based access controls
• Implement data models and transformation logic in alignment with enterprise data architecture standards.
• Ensure data integrity, accuracy, and consistency across source and target systems.
• Collaborate with Data Architects, Business Analysts, and Solution Architects in designing scalable data solutions.
• Support analytics, reporting, dashboards, and AI/ML initiatives by providing trusted and governed datasets.
• Monitor and troubleshoot data pipelines, integration jobs, and platform performance issues.
• Implement data governance, security, privacy, and compliance requirements within data solutions.
• Maintain technical documentation, data dictionaries, and data lineage artifacts.
• Identify opportunities for automation and continuous improvement in data engineering processes.

Profile

• Bachelor's Degree in Computer Science, Information Technology, Software Engineering, Data Science, or related discipline.
• Minimum 8-12 years of experience in data engineering, data integration, or related fields.
• Strong understanding of data warehousing, data lakes, and modern data architecture principles.
• Experience with ETL/ELT tools and data integration platforms.
• Proficiency in SQL and experience with Python, Scala, or similar programming languages.
• Hands-on experience with cloud data platforms such as AWS, Azure, or Google Cloud.
• Experience integrating enterprise applications such as SAP, Salesforce, CRM, ERP, or similar business systems.
• Knowledge of data modeling concepts, including dimensional and relational modeling.
• Familiarity with data governance, data quality, and information security practices.
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and stakeholder management capabilities.
• Preferred Qualifications
• Experience with SAP data structures and Salesforce data models.
• Experience in AWS services such as S3, Glue, Redshift, Lambda, or equivalent cloud technologies.
• Exposure to AI/ML data preparation and feature engineering.
• Knowledge of master data management (MDM) and enterprise data governance.
• Professional certifications in cloud, data engineering, or analytics technologies would be an advantage.