Azure Data Engineer - Databricks & Data Pipeline Expert

E-Outsource Asia; Kuala Lumpur; On-site; MYR 150,000 - 230,000; Full time

Job summary

E-Outsource Asia in Kuala Lumpur is seeking a Senior Data Engineer to design, build and optimize data pipelines on Azure Databricks. You will lead data integration across batch and streaming sources, create curated analytics datasets and ensure governance and quality across the platform.

You'll work with cross-functional teams, implement CI/CD for data jobs, and contribute to platform roadmaps while maintaining SLAs/SLOs. Fluent English and Mandarin helpful for client engagement.

Qualifications

• 3+ to 5+ years of delivery and design leadership.
• Hands-on Databricks experience (PySpark/SQL).
• Strong SQL and production Python experience.
• Experience with Azure data services (ADF, ADLS Gen2).
• Familiarity with Delta Lake and Lakehouse architectures.
• CI/CD and data pipeline automation experience.
• Fluent English and Mandarin for client communication.

Responsibilities

• Design, develop, test, deploy and maintain ETL/ELT pipelines on Azure.
• Ingest and integrate data from batch and streaming sources into ADLS Gen2.
• Build curated datasets for analytics and ML.
• Implement data quality checks, tests, and validation gates.
• Monitor pipelines, logging, alerting and RCA.
• Optimize pipeline performance and cost.
• Implement CI/CD for data jobs and infra.
• Ensure data governance, lineage and security; register assets.
• Collaborate with cross-functional teams; define SLAs/SLOs.
• Contribute to runbooks, docs, coding standards, and platform roadmap.

Skills

Databricks

PySpark

SQL

Python

Azure Data Factory

ADLS Gen2

Delta Lake

Lakehouse

CI/CD

Data governance

Data quality

Power BI

Looker

Tableau

Purview

English

Mandarin

Education

Bachelor’s degree in Information Systems / Computer Science / Data Management or related field

Job description

E-Outsource Asia in Kuala Lumpur is seeking a Senior Data Engineer to design, build and optimize data pipelines on Azure Databricks. You will lead data integration across batch and streaming sources, create curated analytics datasets and ensure governance and quality across the platform.

You'll work with cross-functional teams, implement CI/CD for data jobs, and contribute to platform roadmaps while maintaining SLAs/SLOs. Fluent English and Mandarin helpful for client engagement.