Senior Data Engineer

Inter Island Group; Selangor; On-site; MYR 120,000 - 180,000; Full time

Job summary

Inter Island Group in Malaysia is seeking an experienced Data Engineer to join the Automotive Industry, building modern cloud-based data solutions on Microsoft Azure and Azure Databricks.

You will develop and maintain data pipelines with Azure Databricks, Python, Spark, and manage Databricks workflows, ADLS, Hive Metastore, and compute clusters, while implementing CI/CD with Azure DevOps and GitHub Actions.

Qualifications

• 7+ years of data engineering experience.
• Hands-on experience with Azure Databricks, Python, and Azure DevOps.
• Strong knowledge of Spark and SQL.
• Experience with Microsoft Azure components (ADLS, Key Vault, Entra ID).
• CI/CD, GitHub/Azure Repos, and YAML pipelines.
• Experience with FastAPI, REST APIs, and Docker.
• Cloud-native data engineering mindset.

Responsibilities

• Develop and maintain data pipelines using Azure Databricks, Python, and Spark.
• Manage Databricks workflows, ADLS, Hive Metastore, and compute clusters.
• Develop and maintain CI/CD pipelines using Azure DevOps and GitHub Actions.
• Work with Azure Key Vault, Entra ID, and Azure Log Analytics.
• Develop and integrate REST APIs using FastAPI.
• Build and deploy containerized applications using Docker and Azure Container Registry.
• Work with NATS and Redis for messaging and caching.
• Collaborate with project teams using SCRUM.

Skills

Data engineering

Scrum

Tools

Azure Databricks

Python

Spark

Azure DevOps

GitHub Actions

Azure Key Vault

Docker

FastAPI

REST APIs

NATS

Redis

Azure Container Registry

SQL

Entra ID

ADLS

Hive Metastore

Job description

Jora Malaysia will close on 16th September 2026. Thank you for being with us, we are cheering you on as you continue your career journey.

We are looking for an experienced Data Engineer to join Automotive Industry and support the development of modern cloud-based data solutions using Microsoft Azure and Azure Databricks.

Responsibilities

• Develop and maintain data pipelines using Azure Databricks, Python, and Spark.
• Manage Databricks workflows, ADLS, Hive Metastore, and compute clusters.
• Develop and maintain CI/CD pipelines using Azure DevOps and GitHub Actions.
• Work with Azure Key Vault, Entra ID, and Azure Log Analytics.
• Develop and integrate REST APIs using FastAPI.
• Build and deploy containerized applications using Docker and Azure Container Registry.
• Work with NATS and Redis for messaging and caching.
• Collaborate with project teams using SCRUM.

Requirements

• 7+ years of experience in Data Engineering.
• 7+ years of hands-on experience with Azure Databricks, Python, and Azure DevOps.
• Strong knowledge of Apache Spark and SQL.
• Experience with Microsoft Azure, ADLS, Azure Key Vault, and Entra ID.
• Experience with CI/CD, GitHub/Azure Repos, and YAML pipelines.
• Experience with FastAPI, REST APIs, and Docker.
• Strong understanding of cloud-native data engineering.

Good to Have: Unity Catalog, Medallion Architecture, Pandas/Polars, KQL, PowerShell/Bash, SAFe, NATS, and Redis.