Senior Data Engineer

XENERA MALAYSIA SDN. BHD.; Petaling Jaya; On-site; MYR 96,000 - 144,000; Full time

Job summary

A leading technology company in Malaysia is seeking a Data Management Engineer to develop and manage ETL processes. The ideal candidate has over 3 years of big data development experience, expertise in Python, and familiarity with data warehouses. Responsibilities include optimizing ETL performance and supporting business intelligence solutions. This role demands strong communication skills and the ability to work with large datasets.

Qualifications

• 3+ years of big data development experience handling terabyte-scale datasets.
• Strong English communication skills required.
• Mastery in at least one relational database like MySQL or PostgreSQL.

Responsibilities

• Develop, monitor, and maintain ETL processes for data loading.
• Optimize ETL performance for efficient data processing.
• Provide day-to-day support for BI and Data Management solutions.

Skills

Big data development experience

Python programming

SQL scripting

Apache Airflow

SparkSQL

ETL/ELT processes

Data warehouses knowledge

Git version control

Education

Bachelor's degree in Computer Science or related field

Tools

AWS Redshift

Snowflake

Databricks

Docker

Job description

Overview

Data Management: Develop, monitor, and maintain ETL (Extract, Transform, Load) processes to ensure data is accurately and efficiently loaded into data warehouses or data marts.

Work with BI Architects to design logical and physical analytical data models.

Provide day-to-day support for BI and Data Management solutions.

Create and maintain data dictionaries and data lineage. Resolve data quality issues and establish data integrity checks.

Performance Tuning: Optimize ETL performance to ensure efficient data processing and reporting.

Qualifications

Must-have:

• Bachelor's degree or above in Computer Science, Statistics, Mathematics, or a related STEM field.
• 3+ years of big data development experience, handling terabyte-scale datasets.
• Strong English communication skills in all areas (reading, writing, speaking, and listening).
• Proficient in the Python programming language and common frameworks, with extensive experience using libraries such as NumPy, Pandas, scikit-learn, and Matplotlib/Plotly.
• Solid understanding of core big data ETL/ELT processes and data layering concepts, with hands-on experience in designing and building ETL/ELT pipelines.
• Highly skilled in developing and deploying solutions with Apache Airflow and SparkSQL.
• Familiar with data warehouses (e.g., AWS Redshift, Snowflake) and have development/configuration experience with services/tools such as AWS Glue, Step Functions, Athena, Databricks, or dbt.
• Mastery of at least one relational database like MySQL or PostgreSQL, with strong SQL scripting and performance tuning skills.
• Proficient in using Git for version control, with a commitment to writing clean, well-documented code.

Nice-to-have:

• Hands-on experience in developing and configuring platforms such as Databricks and dbt.
• Experience with SAP data integration is a strong plus.
• Familiarity with at least one mainstream BI platform like Tableau, Microsoft Power BI, or AWS QuickSight.
• Knowledge of Redis, Elasticsearch, and related technologies.
• Proficient in Docker container technology, with the ability to independently build Docker images and deploy services via containerization.

Additional Preferred Qualifications:

• Strong logical thinking skills, fast learning agility, data sensitivity, and a strong attention to detail.
• Ability to explore and clarify business processes through data analysis.
• Customer-oriented mindset with excellent communication skills.

Notes

Removed non-essential boilerplate and reorganized content into clearly defined sections. This version preserves the original job information while improving readability and structure.