Databricks Data Engineer: Build Scalable Azure Pipelines

ITCAN PTE. LIMITED; Singapore; On-site; SGD 70,000 - 110,000; Full time

Job summary

ITCAN PTE. LIMITED is seeking a Data Engineer to design, develop, and maintain scalable data pipelines on Databricks and cloud platforms.

You will integrate diverse data sources and support analytics, reporting, and ML workloads while upholding governance and reliability. You will collaborate with analytics, product, and infrastructure teams to advance the data platform, implement monitoring, and optimise performance using Azure, Delta Lake, and related tools.

Qualifications

• 3+ years of data engineering experience with scalable pipelines.
• Experience with PySpark, Spark SQL, Databricks notebooks/jobs.
• Experience with Azure data services and orchestrators like ADF or Airflow.

Responsibilities

• Design, develop, and maintain scalable data pipelines on Databricks and cloud platforms.
• Integrate data from databases, APIs, log files, and streaming platforms.
• Develop data transformation routines to clean, normalize, and aggregate data.
• Implement data governance in alignment with company standards.
• Collaborate with analytics, product, and infrastructure teams to enhance data platform.

Skills

Data engineering

ETL pipelines

PySpark

SQL

Databricks

Azure

Airflow

CI/CD

Scrum

Kafka/Flink

Tools

ADF

Airflow

Git

Job description

ITCAN PTE. LIMITED is seeking a Data Engineer to design, develop, and maintain scalable data pipelines on Databricks and cloud platforms.

You will integrate diverse data sources and support analytics, reporting, and ML workloads while upholding governance and reliability. You will collaborate with analytics, product, and infrastructure teams to advance the data platform, implement monitoring, and optimise performance using Azure, Delta Lake, and related tools.