Senior Data Engineer

PERSOL; Kuala Lumpur; On-site; MYR 60,000 - 90,000; Full time

Job summary

A leading recruitment agency is looking for a Senior Data Engineer in Kuala Lumpur. You will design and maintain data infrastructure and collaborate with business stakeholders to support analytics and AI initiatives. The ideal candidate has significant experience in data engineering and proficiency with key technologies including SQL, Python, and cloud services. This role is critical for data-driven decision-making across the organization.

Qualifications

• 5+ years experience in data engineering or PhD with 2+ years experience.
• Proficiency in data engineering tools and technologies.
• Experience with cloud platforms and services.

Responsibilities

• Design and maintain data infrastructure and pipelines.
• Collaborate with stakeholders to fulfill data requirements.
• Implement robust data architectures and pipelines.

Skills

SQL

Python

Spark

Kafka

Data Modeling

Data Quality

Communication

Education

Bachelor's Degree

Master's Degree

PhD

Tools

AWS

Azure

Google Cloud

Palantir Foundry

Qlik

Power BI

Git

Job description

The Senior Data Engineer will be responsible for designing, building, and maintaining the data infrastructure and pipelines that support the organization’s analytics and AI initiatives. This role involves collaborating with business stakeholders to ensure data accessibility, quality, and reliability. The Data Engineering function will enable data-driven decision-making across the organization and support affiliates in the APAC and MEA regions.

Responsibilities

• Work closely with ML Engineers, WWD business partners, analysts, and business stakeholders to fulfill data needs and requirements.
• Translate requirements into technical specifications for data engineering projects, working with data analysts and the business.
• Data Architecture: Implement robust, scalable data architectures and pipelines.
• Pipeline Development and Management: Build and maintain ETL (Extract, Transform, Load) processes to integrate data from various sources.
• Ensure data pipelines are reliable, efficient, and scalable; monitor and optimize performance; troubleshoot issues to ensure continuous data availability and reliability.
• Proactively manage data automation to reduce manual efforts in data management platforms and visualization tools.
• Data Quality and Governance: Implement data quality checks and validation processes to ensure data accuracy and consistency.
• Ensure compliance with data policies and regulations.
• Establish and maintain data documentation and metadata management practices.
• Research and Innovation: Stay up to date with the latest trends and technologies in data engineering and big data; evaluate and implement new tools and technologies; promote best practices and continuous improvement.

Experience Required

• Bachelor’s Degree with 5+ years’ experience; or Master’s degree with 5+ years’ experience; or PhD with 2+ years’ experience.

Knowledge / Education Required

• Proficiency in data engineering tools and technologies (e.g., SQL, Python, Spark, Kafka).
• Proven data engineering experience adhering to best practices: designing, building, operationalizing, securing, and monitoring data pipelines and data stores.
• Experience with cloud platforms and services (e.g., Palantir Foundry, AWS, Azure, Google Cloud).
• Experience with DevOps principles and version control (Git preferred).
• Demonstrated experience in Data Management best practices, including Data Lifecycle Management across Core IT & Big Data ecosystems as well as Data Privacy & Security constraints.
• Strong knowledge of architecture methodologies, principles, and frameworks.
• Knowledge and experience in Data Modelling.
• Preferably experience in visualization technologies like Qlik and Power BI or web development.
• Preferably experience in machine learning concepts, algorithms, and deployment strategies.
• Excellent problem-solving skills and a proactive approach to addressing technical challenges.
• Strong communication skills (fluent in English) and the ability to collaborate effectively across different teams.
• Good communication and presentation skills, able to explain complex problems and solutions to a non-technical audience and present technical solutions.
• Strong business acumen and understanding of how data can drive business value.

Be careful not to provide bank or credit card details when applying for jobs. Do not transfer money or complete suspicious online surveys. If you see something suspicious, report this job ad.