Senior Data Engineer | Azure, PySpark & Data Observability

NOVACLOUD SYSTEMS PTE. LTD.; Singapore; On-site; SGD 90,000 - 150,000; Full time

Job summary

NOVACLOUD SYSTEMS PTE. LTD. is seeking an experienced Data Engineer to design and maintain data pipelines for business and analytics.

You will leverage Azure Data Factory, Azure Databricks, PySpark, and SQL to process and prepare data, while integrating Delta Lake and Google BigQuery for analytics storage. The role emphasizes data observability, API integrations, and developing Power BI dashboards to drive insights.

Qualifications

• 4+ years of professional experience in data engineering or analytics.
• Strong hands-on experience with SQL and Azure Databricks.
• Proficient in PySpark, Python, and Azure Data Factory.
• Experience with Delta Lake, REST API, JSON, and Google BigQuery.
• Experience with cloud-based data platforms and scalable pipelines.
• Knowledge of data observability and data reliability practices.
• Experience developing Power BI dashboards.
• Hands-on experience with JMeter for API/load testing.
• Strong analytical, troubleshooting, and problem-solving skills.
• Good communication and cross-functional collaboration.

Responsibilities

• Develop and maintain reliable data pipelines for business and analytics.
• Use Azure Data Factory and Azure Databricks for ingestion and processing.
• Write optimized SQL, Python, and PySpark code for data prep.
• Work with Delta Lake and Google BigQuery for data storage.
• Monitor data processes and apply data observability practices.
• Build and maintain REST API integrations between apps.
• Process and validate JSON data in data workflows.
• Support automated build and deployment via Azure DevOps.
• Develop Power BI dashboards and reports.
• Perform API/load testing with JMeter.
• Investigate issues and coordinate with teams to resolve data challenges.

Skills

Analytical thinking

Troubleshooting

Communication

Cross-functional collaboration

Education

Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or related field

Tools

SQL

Azure Databricks

Python

PySpark

Azure Data Factory

Delta Lake

Google BigQuery

REST API

JSON

Azure DevOps

Power BI

JMeter

ETL

Job description

NOVACLOUD SYSTEMS PTE. LTD. is seeking an experienced Data Engineer to design and maintain data pipelines for business and analytics.

You will leverage Azure Data Factory, Azure Databricks, PySpark, and SQL to process and prepare data, while integrating Delta Lake and Google BigQuery for analytics storage. The role emphasizes data observability, API integrations, and developing Power BI dashboards to drive insights.