Senior Data Engineer

WE Asia Pte Ltd; Kuala Lumpur; On-site; MYR 180,000 - 240,000; Full time

Job summary

WE Asia Pte Ltd in Kuala Lumpur is seeking a Senior Data Engineer to lead data modelling and pipeline development to meet business requirements. You will work with the big data team, optimise data pipelines, and produce MIS reports and dashboards using SAP and Power BI.

The role requires hands-on experience with AWS services (Redshift, Glue, S3, Lambda, RDS, Aurora, API Gateway), strong Python/PySpark/SQL, and excellent stakeholder communication.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or related field; Master's preferred.
• 5+ years hands-on experience in data integration and analytics in a corporate or consulting setting.
• Design, develop, and maintain efficient data pipelines to support data processing, transformation, and integration needs.
• Collaborate with cross-functional teams to understand data requirements and implement solutions that address business needs.
• Work with various data sources and formats, ensuring data quality and consistency.
• Monitor and troubleshoot data pipelines to ensure consistent data flow.
• Proficiency in Python, PySpark, SQL, and Linux.
• Strong ETL/ELT skills, including architecture modelling, data modelling, dashboard building, storytelling, and documentation.
• Strong analytical and problem-solving skills with the ability to translate business requirements into technical solutions.
• Excellent communication and stakeholder engagement skills.

Responsibilities

• Discuss with Solution Architect on designing data pipelines.
• Develop data pipelines with proposed solution.
• Manage and maintain data pipelines.
• Develop/Enhance SAP reports based on business requirements.
• Develop/Enhance Dashboards based on business requirements.
• Fix bug whenever detected.
• Deploy changes into GitHub.
• Create documentation for data pipelines, data models and challenges faced in BAU tasks.

Skills

Data modelling

Data pipeline development

Python

PySpark

SQL

Linux

ETL/ELT

Power BI

SAP

Git/GitHub

Communication with stakeholders

Education

Bachelor's degree in CS/Engineering

Master's degree preferred

Tools

AWS Redshift

AWS Glue

AWS S3

AWS Lambda

AWS RDS

AWS Aurora

AWS API Gateway

Power BI

SAP

Job description

We are looking for a skilled Senior Data Engineer with expertise in data modelling and data pipeline development. As a Senior Data Engineer, you will work with the big data team, adopting best practices to develop and optimise data pipelines, reporting, and dashboards to meet business requirements. Proficiency in serverless cloud services such as AWS Glue, Aurora, Lambda, RDS, and Redshift is essential. You will also handle Management Information Systems (MIS) to generate reports and develop dashboards using business intelligence tools including SAP and Power BI.

• Discuss with Solution Architect on designing data pipelines.
• Develop data pipelines with proposed solution.
• Manage and maintain data pipelines.

Data Modeling

• Create and maintain data models, including entity-relationship diagrams, to represent data structures and relationships.
• Design database schemas that optimize data storage, retrieval and query performance.

Reporting and Visualization

• Communicate effectively with stakeholders.
• Engage with business users to understand reporting needs.
• Deliver timely and accurate reports and dashboards.

BAU Tasks

• Understanding and translating business requirements into technical requirements.
• Develop/Enhance SAP reports based on business requirements.
• Develop/Enhance Dashboards based on business requirements.
• Fix bug whenever detected.
• Deploy changes into GitHub.

Documentation

• Create documentation for data pipelines, data models and challenges faced in BAU tasks.
• Bachelor's Degree in Computer Science, Engineering, or a related field; a Master's degree is preferred.
• 5+ years of hands-on experience in data integration and analytics in a corporate or consulting setting.
• Design, develop, and maintain efficient data pipelines to support data processing, transformation, and integration needs.
• Collaborate with cross-functional teams to understand data requirements and implement solutions that address business needs.
• Work with various data sources and formats, ensuring data quality and consistency.
• Monitor and troubleshoot data pipelines to ensure consistent data flow.
• Handle Management Information Systems (MIS), including data collection, analysis, and reporting for decision-making purposes.
• Able to build reports using tools such as SAP, Power BI, or other reputable BI tools.
• Hands-on experience in AWS services (AWS Redshift, AWS Glue, AWS S3, AWS Lambda, AWS RDS, AWS Aurora, AWS API Gateway).
• Proficient in Python, PySpark, SQL, and Linux.
• Strong ETL/ELT skills, including architecture modelling, data modelling, dashboard building, storytelling, and documentation.
• Strong analytical and problem-solving skills with the ability to translate business requirements into technical solutions.
• Excellent communication and stakeholder engagement skills.
• Proactive, self-driven, and able to work independently.
• Strong attention to data quality and consistency.
• Proficiency in Git and GitHub for version control.