Databricks Data Engineer - Build Scalable ETL/Delta Lake

Avanade Inc.; Kuala Lumpur; On-site; MYR 150,000 - 200,000; Full time

Job summary

A global technology consulting firm in Kuala Lumpur seeks analytical data professionals to join their Data & AI Practice. Responsibilities include designing and maintaining ETL/ELT pipelines using Databricks, optimizing performance, and ensuring data quality. Ideal candidates will work with cross-functional teams to support analytics and machine learning, implementing CI/CD for automation and ensuring documentation for processes. Join a team that values data-driven decision-making and problem-solving.

Qualifications

• Proficiency in designing and maintaining ETL/ELT pipelines.
• Experience with Databricks and Spark job optimization.
• Familiar with data quality checks and governance policies.

Responsibilities

• Design, build, and maintain scalable data pipelines.
• Utilize Databricks features for data orchestration.
• Implement CI/CD and DataOps for automated deployment.

Skills

Data Pipeline Development

ETL/ELT pipelines

Databricks

PySpark

SQL

Data governance

Tools

Databricks Workflows

Delta Lake

Job description

A global technology consulting firm in Kuala Lumpur seeks analytical data professionals to join their Data & AI Practice. Responsibilities include designing and maintaining ETL/ELT pipelines using Databricks, optimizing performance, and ensuring data quality. Ideal candidates will work with cross-functional teams to support analytics and machine learning, implementing CI/CD for automation and ensuring documentation for processes. Join a team that values data-driven decision-making and problem-solving.