Data Engineer

WE Asia Pte Ltd; Kampung Malaysia Tambahan; Sur place; MYR 150 000 - 230 000; Plein temps

Résumé du poste

WE Asia Pte Ltd is seeking a skilled Data Engineer in Kuala Lumpur to design and build data pipelines, models and MIS reports. You will work with the big data team to implement serverless cloud services (AWS Glue, Redshift, Lambda, RDS, Aurora) and to develop dashboards using SAP and Power BI.

You will collaborate with Solution Architects and business users to deliver timely reports and robust data solutions.

Qualifications

• Bachelor's degree in CS/Engineering or related field; Master's preferred.
• 5+ years hands-on data integration and analytics experience.
• Design, develop and maintain efficient data pipelines.
• Experience with SAP, Power BI and MIS reporting.
• Proficient in Python, SQL, PySpark and Linux.
• Strong ETL/ELT, data modelling and dashboard building skills.
• Ability to translate business requirements into technical solutions.
• Excellent communication and stakeholder engagement.

Responsabilités

• Discuss with Solution Architect on designing data pipelines.
• Develop data pipelines with proposed solution.
• Manage and maintain data pipelines.
• Create and maintain data models and schemas.
• Develop/Enhance SAP reports and dashboards per requirements.
• Translate business needs into technical specifications.
• Fix bugs and deploy changes into GitHub.
• Document data pipelines, models, and challenges.

Connaissances

Data modelling

Data pipelines

AWS

PySpark

SQL

Python

Linux

ETL/ELT

Dashboard design

Data quality

Stakeholder comms

Formation

Bachelor's degree in CS/Engineering or related field

Master's degree preferred

Outils

SAP

Power BI

GitHub

Git

AWS Redshift

AWS Glue

AWS S3

AWS Lambda

AWS RDS

AWS Aurora

API Gateway

Description du poste

We are looking for a skilled Data Engineer with expertise in data modelling and data pipeline development. As a Data Engineer, you will work with the big data team, adopting best practices to develop and optimise data pipelines, reporting, and dashboards to meet business requirements. Proficiency in serverless cloud services such as AWS Glue, Aurora, Lambda, RDS, and Redshift is essential. You will also handle Management Information Systems (MIS) to generate reports and develop dashboards using business intelligence tools including SAP and Power BI.

• Discuss with Solution Architect on designing data pipelines.
• Develop data pipelines with proposed solution.
• Manage and maintain data pipelines.

Data Modeling

• Create and maintain data models, including entity-relationship diagrams, to represent data structures and relationships.
• Design database schemas that optimize data storage, retrieval and query performance.

Reporting and Visualization

• Communicate effectively with stakeholders.
• Engage with business users to understand reporting needs.
• Deliver timely and accurate reports and dashboards.

BAU Tasks

• Understanding and translating business requirements into technical requirements.
• Develop/Enhance SAP reports based on business requirements.
• Develop/Enhance Dashboards based on business requirements.
• Fix bug whenever detected.
• Deploy changes into GitHub.

Documentation

• Create documentation for data pipelines, data models and challenges faced in BAU tasks.
• Bachelor's Degree in Computer Science, Engineering, or a related field; a Master's degree is preferred.
• 5+ years of hands-on experience in data integration and analytics in a corporate or consulting setting.
• Design, develop, and maintain efficient data pipelines to support data processing, transformation, and integration needs.
• Collaborate with cross-functional teams to understand data requirements and implement solutions that address business needs.
• Work with various data sources and formats, ensuring data quality and consistency.
• Monitor and troubleshoot data pipelines to ensure consistent data flow.
• Handle Management Information Systems (MIS), including data collection, analysis, and reporting for decision-making purposes.
• Able to build reports using tools such as SAP, Power BI, or other reputable BI tools.
• Hands-on experience in AWS services (AWS Redshift, AWS Glue, AWS S3, AWS Lambda, AWS RDS, AWS Aurora, AWS API Gateway).
• Proficient in Python, PySpark, SQL, and Linux.
• Strong ETL/ELT skills, including architecture modelling, data modelling, dashboard building, storytelling, and documentation.
• Strong analytical and problem-solving skills with the ability to translate business requirements into technical solutions.
• Excellent communication and stakeholder engagement skills.
• Proactive, self-driven, and able to work independently.
• Strong attention to data quality and consistency.
• Proficiency in Git and GitHub for version control.