Data Engineer - Microsoft Fabrics

VISEO ASIA PTE. LTD.; Singapore; On-site; SGD 80,000 - 110,000; Full time

Job summary

A leading technology consulting company in Singapore seeks a Senior Data Engineer to design and maintain scalable data infrastructure and pipelines. The ideal candidate has over 6 years of experience in data engineering, proficiency in SQL and Python, and expertise in big data technologies. This role involves collaborating with various teams to drive data quality and performance. Competitive compensation and a focus on innovation are offered.

Qualifications

• 6+ years of experience in a relevant field.
• Strong understanding of data security and compliance (e.g., GDPR, HIPAA).

Responsibilities

• Design, build, and maintain robust, scalable data pipelines.
• Develop and optimize data models and ETL processes.
• Collaborate with data teams to understand data requirements.
• Ensure data integrity and enforce governance policies.
• Monitor and optimize data processing speed and query performance.
• Guide and mentor junior engineers.
• Stay updated on emerging technologies and lead integration initiatives.

Skills

SQL

Python

ETL/ELT tools

Big data technologies (Hadoop, Spark)

Data modelling

Data warehousing

Stream processing

Data security

Compliance

Governance

CI/CD pipelines using Azure DevOps

Education

Bachelor's or Master's in computer science, data science, information technology, or related field

Tools

Apache Airflow

Kafka

Spark

Azure Data Factory

Microsoft Fabric

Azure Synapse Analytics

Azure Databricks

Power BI

Azure SQL Database/ SQL Server

Azure Purview

Platforms (Azure, AWS, GCP)

Job description

Job Description & Requirements

The Senior Data Engineer plays a criticalrole in designing, developing, and maintaining scalable data infrastructure and pipelines. This position supports data-driven decision-making across the organization by ensuring data quality, accessibility, and performance.

Roles and Responsibilities

• Data Pipeline Development: Design, build, and maintain robust, scalable data pipelines using tools like Apache Airflow, Kafka, and Spark.
• Data Architecture: Develop and optimizedata models, data warehouses, and ETL processes to support analytics and machine learning initiatives.
• Collaboration: Work closelywith data scientists, analysts, and software engineers to understand data requirements and deliver tailored solutions.
• Data ǪualityG Governance: Ensure data integrity, implement validation checks, and enforce data governance policies.
• Performance Optimization: Monitor and improvedata processing speed,system uptime, and query performance.
• Leadership G Mentorship: Guide junior engineers, promote best practices, and foster a culture of continuous learning.
• Innovation G Research: Stay updated with emerging technologies in AI, automation, and data streaming; lead initiatives to integrate them into existing systems.

Profile

• Bachelor's or Master's in computer science, data science, information technology, or a related field, with 6+ years of experience.
• Proficiency in SǪL, Python, and ETL/ELT tools
• Experience in Microsoft solutions are as follows:
• Azure Data Factory
• Microsoft Fabric
• Azure Synapse Analytics
• Azure Databricks
• Power BI
• Azure SǪL Database/ SǪL Server
• Azure Purview
• Experience with big data technologies (Hadoop, Spark)
• Familiarity with platforms(Azure, AWS, and GCP)
• Strong understanding of data modelling, data warehousing, and stream processing
• Knowledge of data security, compliance, and governance
• Knowledge of CI/CD pipelines using Azure DevOps
• Understanding of data security and compliance(e.g., GDPR, HIPAA)