Data Engineer

Pentas Flora Management Services Sdn Bhd; Kuala Lumpur; In loco; MYR 90.000 - 180.000; Tempo pieno

Descrizione del lavoro

PENTAS FLORA is seeking a Data Engineer to manage and modernise its enterprise data integration and data warehouse environment. You will develop pipelines to extract data from Core System and other business systems, transform and validate it with Python and ETL tools, and load into Microsoft SQL Server for reporting and analytics.

The role requires a Bachelor's degree and at least 3 years of hands-on experience in data engineering, ETL or data warehousing, with strong SQL Server, Python, and

Competenze

• Bachelor's degree in computer science, IT, data engineering or a related discipline.
• Minimum 3 years of hands-on data engineering, ETL or data warehousing experience.
• Strong practical experience with Microsoft SQL Server, T-SQL and Python-based data processing.
• Strong Python capability for production data extraction, transformation and loading.
• Good understanding of dimensional modelling, ETL pipelines, incremental loading and data quality controls.
• Experience with incremental loading, data validation, exception handling and reconciliation.
• Understanding of layered data architecture such as Medallion Architecture or an equivalent approach.

Mansioni

• Develop and maintain Python-based ETL pipelines to extract, transform, validate and load data from Core System and other business systems.
• Implement full and incremental data-loading processes with error handling, logging, reconciliation and recovery mechanisms.
• Monitor scheduled pipelines, troubleshoot issues and ensure reliable data processing.
• Design and maintain a Microsoft SQL Server-based data warehouse using dimensional modelling, fact and dimension tables, star schemas and historical data management.
• Support layered architecture based on Medallion Architecture, or equivalent Raw, Staging, Curated and Business-ready layers.
• Develop and optimise T-SQL queries, views and stored procedures to support efficient data processing.
• Implement data quality controls covering completeness, accuracy, duplicates and source-to-target reconciliation.
• Maintain data dictionaries, data mappings, business rules, data lineage and technical documentation.
• Provide reliable and business-ready datasets for Microsoft Power BI and enterprise reporting.
• Assess the existing Python ETL environment and identify opportunities to improve reliability, scalability and maintainability.

Conoscenze

Python programming

ETL design

Data warehousing

T-SQL

Formazione

Bachelor's Degree in Computer Science / IT / Data Engineering

Strumenti

Microsoft SQL Server

Power BI

Azure Data Factory

SSIS

REST API integration

Descrizione del lavoro

The Data Engineer is responsible for managing, enhancing and progressively modernising Pentas Flora's enterprise data integration and data warehouse environment. You will develop and maintain pipelines that extract operational data from Core System and other business systems, transform and validate it using Python and ETL technologies, and load it into Microsoft SQL Server for reporting and analytics.

Key responsibilities

Develop and maintain Python-based ETL pipelines to extract, transform, validate and load data from Core System and other business systems.

Implement full and incremental data-loading processes with appropriate error handling, logging, reconciliation and recovery mechanisms.

Monitor scheduled pipelines, troubleshoot issues and ensure reliable data processing.

Design and maintain a Microsoft SQL Server-based data warehouse using dimensional modelling, fact and dimension tables, star schemas and historical data management.

Support layered architecture based on Medallion Architecture, or equivalent Raw, Staging, Curated and Business-ready layers.

Develop and optimise T-SQL queries, views and stored procedures to support efficient data processing and reporting.

Implement data quality controls covering completeness, accuracy, duplicates and source-to-target reconciliation.

Maintain data dictionaries, data mappings, business rules, data lineage and technical documentation.

Provide reliable and business-ready datasets for Microsoft Power BI and enterprise reporting.

Assess the existing Python ETL environment and identify opportunities to improve reliability, scalability and maintainability.

About you

Bachelor's Degree in Computer Science, Information Technology, Data Engineering or a relevant discipline.

Minimum 3 years of hands-on experience in data engineering, ETL or data warehousing.

Strong practical experience with Microsoft SQL Server, T-SQL and Python-based data processing.

Strong Python capability for production data extraction, transformation and loading.

Good understanding of dimensional modelling, ETL pipelines, incremental loading and data quality controls.

Practical experience with incremental loading, data validation, exception handling and reconciliation.

Understanding of layered data architecture such as Medallion Architecture or an equivalent approach.

Ability to troubleshoot data discrepancies and translate business reporting requirements into data solutions.

Experience with Microsoft Power BI, Microsoft Fabric, Azure Data Factory, SSIS, REST API integration or cloud data platforms is an added advantage.

Unlock job insights

Hirer responsiveness Salary match Number of applicants

Your application will include the following questions:

• Which of the following statements best describes your right to work in Malaysia?
• What's your expected monthly basic salary?
• How many years' experience do you have as a Data Engineer?
• Which of the following types of qualifications do you have?
• Do you have experience using T-SQL?

PENTAS FLORA is a leading scheduled waste management company committed to sustainable resource recovery. We specialise in transforming hazardous waste streams into valuable resources, producing high-quality re-refined base oil, fuel oil, and a range of other reusable materials. Dedicated to environmental stewardship, we leverage innovation and cutting-edge technology to minimise environmental impact and maximise resource value. Our 'cradle to cradle' philosophy underpins our commitment to responsible waste management practices, driving a cleaner, greener future. Partner with us to contribute to a more sustainable world.

PENTAS FLORA is a leading scheduled waste management company committed to sustainable resource recovery. We specialise in transforming hazardous waste streams into valuable resources, producing high-quality re-refined base oil, fuel oil, and a range of other reusable materials. Dedicated to environmental stewardship, we leverage innovation and cutting-edge technology to minimise environmental impact and maximise resource value. Our 'cradle to cradle' philosophy underpins our commitment to responsible waste management practices, driving a cleaner, greener future. Partner with us to contribute to a more sustainable world.