Senior Azure Data Engineer

Agensi Pekerjaan RecruitFirst Sdn Bhd; Kuala Lumpur; On-site; MYR 120,000 - 210,000; Full time

Job summary

Agensi Pekerjaan RecruitFirst Sdn Bhd in Kuala Lumpur seeks an experienced Data Engineer to design, build, and optimize scalable data pipelines and analytics solutions across multiple sources. You will work with analysts, data scientists, stakeholders, and vendors to translate requirements into robust data infrastructure using Azure Synapse, Databricks, Python, and PySpark.

This role emphasizes data governance, security, and quality across Bronze to Gold lakehouse layers, plus documentation and

Qualifications

• Bachelor’s degree in Computer Science, Data Science, Statistics, Mathematics or related field.
• Minimum 5 years’ experience in data engineering or related role.
• Strong experience with Azure Synapse, Databricks, Delta Lake, and Azure cloud technologies.
• Proficient in Python, PySpark, SQL/T-SQL, and ETL/data pipeline development.
• Experience with data modelling, data management, ETL, BI, dashboards, and data visualization.
• Strong understanding of data governance, security, quality, and BI best practices.
• Strong analytical and problem-solving skills with the ability to translate business needs into technical solutions.
• Excellent communication and stakeholder management skills, with the ability to work across teams and cultures.
• Able to manage multiple priorities in a fast-paced environment and travel internationally when required.
• Fluent in written and spoken English.

Responsibilities

• Design, develop, and maintain scalable data pipelines, ETL processes, and data integration solutions across multiple sources.
• Translate business requirements into data and analytical solutions in collaboration with analysts, data scientists, stakeholders, and vendors.
• Design and optimize data models, including star schemas, aggregation tables, and curated datasets for reporting and analytics.
• Build and manage data infrastructure using Azure Synapse Analytics, Databricks, and other Azure services.
• Develop and automate data ingestion and transformation using SQL, Python, and PySpark.
• Ensure data quality, consistency, security, and reliability across Bronze, Silver, and Gold lakehouse layers.
• Monitor and optimize data platform performance, scalability, and reliability.
• Contribute to data architecture, engineering standards, and best practices as a subject matter expert.
• Maintain documentation for data pipelines, architecture, data sources, data dictionaries, and the Business Glossary.
• Collaborate with vendor data engineers to deliver data infrastructure and integration solutions aligned with business objectives.
• Improve data maturity, governance, and visibility to enable trusted insights and informed business decisions.

Skills

Analytical skills

Problem solving

Communication

Travel willingness

Education

Bachelor’s degree in Computer Science, Data Science, Statistics, Mathematics

Tools

Azure Synapse

Databricks

Delta Lake

Python

PySpark

SQL/T-SQL

Job description

Design, develop, and maintain scalable data pipelines, ETL processes, and data integration solutions across multiple sources.

Translate business requirements into effective data and analytical solutions in collaboration with analysts, data scientists, stakeholders, and vendors.

Design and optimize data models, including star schemas, aggregation tables, and curated datasets for reporting and analytics.

Build and manage data infrastructure using Azure Synapse Analytics, Databricks, and other Azure services.

Develop and automate data ingestion and transformation using SQL, Python, and PySpark.

Ensure data quality, consistency, security, and reliability across Bronze, Silver, and Gold lakehouse layers.

Monitor and optimize data platform performance, scalability, and reliability.

Contribute to data architecture, engineering standards, and best practices as a subject matter expert.

Maintain documentation for data pipelines, architecture, data sources, data dictionaries, and the Business Glossary.

Collaborate with vendor data engineers to deliver data infrastructure and integration solutions aligned with business objectives.

Improve data maturity, governance, and visibility to enable trusted insights and informed business decisions.

Requirements

Bachelor’s degree in Computer Science, Data Science, Statistics, Mathematics, or a related field.

Minimum 5 years’ experience in data engineering or a related role.

Strong experience with Azure Synapse, Databricks, Delta Lake, and Azure cloud technologies.

Proficient in Python, PySpark, SQL/T-SQL, and ETL/data pipeline development.

Experience with data modelling, data management, ETL, BI, dashboards, and data visualization.

Strong understanding of data governance, security, quality, and BI best practices.

Strong analytical and problem-solving skills with the ability to translate business needs into technical solutions.

Excellent communication and stakeholder management skills, with the ability to work across teams and cultures.

Able to manage multiple priorities in a fast-paced environment and travel internationally when required.

Fluent in written and spoken English.