Senior Data Engineer (Microsoft)

FPT ASIA PACIFIC PTE. LTD.; Singapore; On-site; SGD 70,000 - 90,000; Full time

Job summary

A leading technology company in Singapore is seeking a Data Engineer with over 6 years of experience. The role involves designing and maintaining data pipelines, collaborating with data teams, and ensuring data quality. Candidates should be proficient in SQL and Python, and have experience with big data technologies and cloud platforms like Azure. Competitive compensation offered.

Qualifications

• 6+ years of experience in data-related roles.
• Strong understanding of data governance and compliance.
• Experience in performance optimization in data processing.

Responsibilities

• Design, build, and maintain data pipelines using Apache Airflow, Kafka, and Spark.
• Develop and optimize data models and warehouses for analytics.
• Work with teams to deliver tailored data solutions.
• Ensure data integrity and implement governance policies.
• Monitor and enhance data processing speed and performance.

Skills

SQL

Python

ETL/ELT tools

Big data technologies (Hadoop, Spark)

Cloud platforms (Azure, AWS, GCP)

Data modelling

Data warehousing

Stream processing

Data security

CI/CD pipelines

Education

Bachelor's or master's degree in computer science, data science, or related field

Tools

Azure Data Factory

Azure Synapse Analytics

Azure Databricks

Power BI

Azure SQL Database / SQL Server

Azure Purview

Microsoft Fabric

Job description

Roles and Responsibilities

• Data Pipeline Development: Design, build, and maintain robust, scalable data pipelines using tools like Apache Airflow, Kafka, and Spark.
• Data Architecture: Develop and optimize data models, data warehouses, and ETL processes to support analytics and machine learning initiatives.
• Collaboration: Work closely with data scientists, analysts, and software engineers to understand data requirements and deliver tailored solutions.
• Data Quality & Governance: Ensure data integrity, implement validation checks, and enforce data governance policies.
• Performance Optimization: Monitor and improve data processing speed, system uptime, and query performance.
• Leadership & Mentorship: Guide junior engineers, promote best practices, and foster a culture of continuous learning.
• Innovation & Research: Stay updated with emerging technologies in AI, automation, and data streaming; lead initiatives to integrate them into existing systems.
• Other duties as required.

Desired Skills and Experience

• Bachelor's or master's degree in computer science, data science, information technology, or a related field, with 6+ years of experience.
• Proficiency in SQL, Python, and ETL/ELT tools
• Experience with big data technologies (Hadoop, Spark)
• Familiarity with cloud platforms (Azure, AWS, and GCP)
• Strong understanding of data modelling, data warehousing, and stream processing
• Knowledge of data security, compliance, and governance
• Knowledge of CI/CD pipelines using Azure DevOps
• Understanding of data security and compliance (e.g., GDPR, HIPAA)
• Knowledge of data security and compliance (e.g., GDPR, HIPAA)

Experience in Microsoft solutions are as follows:

• Azure Data Factory
• Microsoft Fabric
• Azure Synapse Analytics
• Azure Databricks
• Power BI
• Azure SQL Database / SQL Server
• Azure Purview