Senior Azure Data Engineer - Databricks & Data Pipelines

INTELLECT MINDS PTE. LTD.; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

INTELLECT MINDS PTE. LTD. is seeking an experienced Azure Data Engineer to design, build and optimize large‑scale data pipelines leveraging Azure Data Factory and Databricks.

The candidate will implement PySpark and SQL solutions, ensure data quality, and support data migration efforts. The role focuses on data validation, performance tuning, and CI/CD practices with Azure DevOps and Git, while collaborating with stakeholders and analysts to deliver reliable data platforms in a fast‑paced

Qualifications

• Bachelor's or Master's degree in CS/IT or related field.
• Microsoft Certified: Azure Data Engineer Associate (Preferred).
• Databricks Certified Data Engineer Associate (Preferred).

Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks.
• Develop ETL/ELT solutions using PySpark and SQL.
• Perform source‑to‑target data validation, reconciliation, and data quality checks.
• Build and optimize data ingestion pipelines into Azure Data Lake Storage (ADLS).
• Develop automated data validation scripts using Python/PySpark.
• Validate incremental and full‑load data processing.
• Perform schema validation, duplicate checks, null checks, and aggregation testing.
• Monitor and optimize pipeline performance to meet SLA requirements.
• Troubleshoot and resolve ETL and data pipeline issues.
• Collaborate with business stakeholders, data analysts, and development teams.
• Maintain CI/CD practices using Azure DevOps and Git.

Skills

Azure Databricks

Azure Data Factory

Azure Data Lake Storage

PySpark

Python

Advanced SQL

ETL Development

Data Pipeline Development

Data Validation

Source‑to‑Target Reconciliation

Data Migration

Performance Tuning

Azure DevOps

Git

Education

Bachelor's or Master's degree in Computer Science, Information Technology, or a related field

Microsoft Certified: Azure Data Engineer Associate (Preferred)

Databricks Certified Data Engineer Associate (Preferred)

Job description

INTELLECT MINDS PTE. LTD. is seeking an experienced Azure Data Engineer to design, build and optimize large‑scale data pipelines leveraging Azure Data Factory and Databricks.

The candidate will implement PySpark and SQL solutions, ensure data quality, and support data migration efforts. The role focuses on data validation, performance tuning, and CI/CD practices with Azure DevOps and Git, while collaborating with stakeholders and analysts to deliver reliable data platforms in a fast‑paced