Senior Data Engineer

XENERA MALAYSIA SDN. BHD.; Petaling Jaya; On-site; MYR 80,000 - 120,000; Full time

Job summary

A technology firm in Selangor is seeking a Data Management Professional to develop and optimize ETL processes. Candidates must have a Bachelor's degree in a STEM field, 3+ years of experience with big data, and proficiency in Python and SQL. The role includes supporting BI solutions and requires strong communication skills. Competitive compensation and opportunities for professional development are offered.

Qualifications

• 3+ years of big data development experience, handling terabyte-scale datasets.
• Strong English communication skills in all areas (reading, writing, speaking, and listening).
• Hands-on experience designing and building ETL/ELT pipelines.
• Strong logical thinking skills and attention to detail.

Responsibilities

• Develop, monitor, and maintain ETL processes for accurate data loading.
• Design logical and physical analytical data models with BI Architects.
• Provide day-to-day support for BI and Data Management solutions.
• Optimize ETL performance for efficient data processing and reporting.

Skills

Big data development

Python programming

ETL/ELT processes

SQL scripting

Apache Airflow

SparkSQL

Git version control

Data warehousing (AWS Redshift, Snowflake)

Data integrity checks

Education

Bachelor's degree or above in Computer Science, Statistics, Mathematics, or related STEM field

Tools

NumPy

Pandas

scikit-learn

Matplotlib

Plotly

Docker

AWS Glue

Databricks

Job description

Data Management: Develop, monitor, and maintain ETL (Extract, Transform, Load) processes to ensure data is accurately and efficiently loaded into data warehouses or data mart

Work with BI Architects to design logical and physical analytical data models.

Provide day-to-day support for BI and Data Management solutions

Create and maintain data dictionaries and data lineage. Resolve data quality issues and establish data integrity checks.

Performance Tuning: Optimize ETL performance to ensure efficient data processing and reporting.

Must-have

• Bachelor's degree or above in Computer Science, Statistics, Mathematics, or a related STEM field.
• 3+ years of big data development experience, handling terabyte-scale datasets.
• Strong English communication skills in all areas (reading, writing, speaking, and listening).
• Proficient in the Python programming language and common frameworks, with extensive experience using libraries such as NumPy, Pandas, scikit-learn, and Matplotlib/Plotly.
• Solid understanding of core big data ETL/ELT processes and data layering concepts, with hands-on experience in designing and building ETL/ELT pipelines.
• Highly skilled in developing and deploying solutions with Apache Airflow and SparkSQL.
• Familiar with data warehouses (e.g., AWS Redshift, Snowflake) and have development/configuration experience with services/tools such as AWS Glue, Step Functions, Athena, Databricks, or dbt.
• Mastery of at least one relational database like MySQL or PostgreSQL, with strong SQL scripting and performance tuning skills.
• Proficient in using Git for version control, with a commitment to writing clean, well-documented code.

Nice-to-have

• Hands-on experience in developing and configuring platforms such as Databricks and dbt.
• Experience with SAP data integration is a strong plus.
• Familiarity with at least one mainstream BI platform like Tableau, Microsoft Power BI, or AWS QuickSight.
• Knowledge of Redis, Elasticsearch, and related technologies.
• Proficient in Docker container technology, with the ability to independently build Docker images and deploy services via containerization.

Additional Preferred Qualifications

• Strong logical thinking skills, fast learning agility, data sensitivity, and a strong attention to detail.
• Ability to explore and clarify business processes through data analysis.
• Customer-oriented mindset with excellent communication skills.

Worldline International (Malaysia) Sdn Bhd