Databricks Data Engineer — Real-Time Pipelines on Azure

NCS; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

NCS is hiring a Data Engineer in Singapore to design, develop, and maintain scalable data pipelines on Databricks and cloud platforms. You will integrate diverse data sources and support analytics, reporting, and ML workloads while upholding governance and reliability standards.

You will collaborate with analytics, product, and infra teams to advance the data platform, implement monitoring, and optimise ETL performance across batch and streaming workloads.

Qualifications

• 3+ years in data engineering with scalable pipelines.
• Experience in data modelling and distributed computing.
• Hands-on with PySpark, Spark SQL and Databricks notebooks/jobs.
• Experience orchestrating data pipelines with ADF, Airflow, or similar tools.
• Experience with real-time and batch data processing.
• Proficiency in SQL and performance optimization.
• DevOps, Git workflows, and CI/CD familiarity.
• Scrum experience and ability to work in Scrum teams.
• Exposure to streaming tech like Kafka, Flink, or Kinesis.
• Databricks Certified Data Engineer Associate/Professional preferred.

Responsibilities

• Develop and maintain ETL pipelines for centralized data storage (Delta Lake).
• Integrate data from databases, APIs, logs, and external providers.
• Design data transformations to clean, normalize, and aggregate data.
• Implement data governance and reliability practices.
• Collaborate with analytics, product, and infra teams to enhance data platform.
• Build dashboards, reports, and data visualizations.
• Monitor pipelines and implement error-handling and alerts.
• Work on cloud-based data platforms, primarily Azure.
• Participate in requirement grooming and UAT for data projects.
• Support production deployment and data validation activities.

Skills

Data modelling

Distributed computing

PySpark

Spark SQL

Databricks

SQL proficiency

DevOps & CI/CD

Scrum methodolgy

Streaming tech

Real-time processing

Tools

Azure

ADF

Airflow

Kafka

Flink

Kinesis

Job description

NCS is hiring a Data Engineer in Singapore to design, develop, and maintain scalable data pipelines on Databricks and cloud platforms. You will integrate diverse data sources and support analytics, reporting, and ML workloads while upholding governance and reliability standards.

You will collaborate with analytics, product, and infra teams to advance the data platform, implement monitoring, and optimise ETL performance across batch and streaming workloads.