data engineer

RecruitFirst Pte Ltd; Malaysia; On-site; MYR 112,000 - 123,000; Full time

Job summary

RecruitFirst Pte Ltd is seeking a Senior Data Engineer to design, build, and optimize data pipelines and analytics infrastructure across Southeast Asia markets. The role emphasizes Azure Synapse Analytics, Databricks, and ETL automation using Python/PySpark.

You will collaborate with cross-functional teams to deliver scalable data solutions, ensure data quality, and implement data models across bronze, silver, and gold layers. Strong communication and stakeholder management are essential.

Qualifications

• Bachelor’s degree required in CS/Data Science/Statistics/Math or equivalent
• Minimum 5 years of experience in data engineering or similar role
• Proven expertise in Azure Synapse Analytics and Databricks
• Proficient in Python and PySpark for ETL automation
• Experience designing data models (star schemas) and data lakes/warehouses
• Experience with Data Management tools (BI/ETL/Dashboards)

Responsibilities

• Design, build, and maintain robust data pipelines and ETL processes across multiple sources
• Develop analytical solutions for strategic business problems via engagement and research
• Collaborate with analysts, scientists, and stakeholders to translate requirements into technical solutions
• Implement and optimize data models (star schemas) for efficient querying
• Leverage Azure Synapse Analytics, Databricks, and related services to scale data infrastructure
• Ensure data quality and governance across data lakehouse layers (bronze-silver-gold)
• Document pipelines and architecture for transparency and knowledge sharing
• Lead and guide data engineering best practices within the organization
• Provide expertise in data integration and data infrastructure across SEA markets

Skills

Python

PySpark

Data analysis

SQL

ETL

Communication

Education

Bachelor’s degree in Computer Science, Data Science, Statistics, Mathematics or equivalent

Tools

Azure Synapse Analytics

Databricks

Job description

Overview

Job-type Full-Time Job Category IT - Software Industries IT Salary MYR 10,000 - 11,000 /Month

Who you'll be working for

The client is a leading F&B company operating over 2,000 outlets.

What requirements you'll need to be eligible

• Bachelor’s degree holder in Computer Science, Data Science, Statistics, Mathematics or equivalent


• Minimum 5 years of experience in data engineering or a similar role.


• Strong proven expertise in Azure Synapse Analytics, Databricks, and other Azure cloud technologies.


• Proficiency in Python, PySpark, and other relevant programming languages for ETL automation.


• Experience in designing and implementing data models, including star schemas and other dimensional models.


• Experience in Data Management tools such as Data Lake, BI, ETL, Dashboards, Balanced Scorecards, etc.


• Excellent communication and presentation skills, to communicate complex analytical and technical content clearly and effectively.


• Demonstrated ability to work with large, complex datasets and build scalable data solutions.


• Track record of forming strong partnerships with cross-functional teams.


• Experience with data visualization tools and an understanding of how data engineering supports data analytics.

KNOWLEDGE, SKILLS & ABILITIES REQUIRED

• Highly proficient in data analysis and data mining skills (such as Python, Scala, R, T-SQL, MDX, DAX, etc.)


• Strong analytical skills with the ability to translate business needs into technical solutions.


• In-depth knowledge and understanding of data relationships and able to comprehend the data.


• Understand the reporting and business intelligence best practices.


• Ability to synthesize information from various sources to Data Management Platform to create a single and actionable point of view and direction for BI reporting requirements.


• Highly resourceful, with strong research skills and a working knowledge of digital marketing analytics.


• Possess a strong ability to query and use large, robust data sets across the latest Delta Lake and Databrick platforms.


• Strong understanding of data governance, security, and compliance best practices.


• Ability to manage multiple priorities and projects simultaneously.

OTHERS

• Fluent communication in English and interpersonal skills, both verbal and written.


• Meticulous and good team player who can work with various levels of stakeholders from different cultures and locations.


• Ability to work in a flexible, dynamic, and fast-paced environment managing multiple projects concurrently.


• Should be able to travel to other countries if needed.

What you'll be doing on the job

We are seeking a skilled Senior Data Engineer to join our team and contribute to the successful implementation of our regional Data & Analytics solutions across Southeast Asia Markets. This role will be pivotal in designing, building, and optimizing our data pipelines and infrastructure, ensuring seamless integration and processing of data from various sources, including CRM, sales, and D365 systems. The ideal candidate will have substantial experience in working with Azure technologies, particularly Databricks and Synapse Analytics, and will be proficient in automating ETL processes using SQL, Python or PySpark.

KEY RESPONSIBILITIES

• Design, develop, and maintain robust data pipelines and ETL processes to support data integration and transformation across multiple data sources.


• Design analytical solutions for strategic business problems through engagement and research.


• Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements and translate them into technical solutions in meeting milestones, timelines, and deliverables for projects.


• Implement and optimize data models, including star schemas, to facilitate efficient data querying and analysis.


• Leverage Azure Synapse Analytics, Databricks, and other Azure services to build scalable data infrastructure.


• Ensure data quality and consistency across all layers of the data lakehouse (bronze, silver, and gold layers).


• Automate ETL processes using SQL, Python, PySpark, or other relevant technologies to streamline data ingestion and transformation.


• Monitor and maintain the performance, security, and reliability of data systems.


• Provide technical leadership and guidance on best practices for data engineering within the organization.


• Document data pipelines, processes, and architecture to ensure transparency and knowledge sharing.


• Work either individually or in project teams as a subject matter expert to deliver business needs.

New data integration and data infrastructure

• A key contributor to the development of organization data infrastructure and collaborate closely with vendor’s data engineers on a day-to-day basis to ensure development drives business outcomes.


• Create impeccable and user-friendly documentation for data sources and data dictionaries across the company.


• Establish a clear Business Glossary to keep everyone on the same page by defining common terminology and interpretation of the same data.


• Validate data structure in the Data Management Platform, refine logic and data transformation, and build aggregation tables to provide seamless reporting and data access across the business.


• Improve data maturity with high data quality, reliability, and visibility of business insights in a consistent manner and enable the business team to identify opportunities and threats for the company.