Data Engineer (Databricks)

EPAM Systems; Kuala Lumpur; On-site; MYR 60,000 - 90,000; Full time

Job summary

EPAM Systems in Kuala Lumpur is seeking an experienced Data Integration Engineer specializing in Databricks to design enterprise data products and pipelines that deliver insights and value to business teams.

You will collaborate with stakeholders, implement ETL/ELT solutions on Azure or AWS, and use SQL, Python, and PySpark to transform large datasets and support Power BI reporting. The role offers exposure to global projects and a path to grow with leading technologies.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or related field.
• Strong experience in data integration and ETL/ELT solutions.
• Proficient in SQL/Python for data transformation and analysis.
• Experience with Databricks and Delta Lake concepts.
• Familiarity with cloud platforms Azure and AWS.
• Hands-on data pipeline development on Databricks and collaboration with analytics teams.
• Excellent written and verbal English communication.

Responsibilities

• Develop, maintain and support ETL/ELT solutions.
• Implement and manage Databricks pipelines/notebooks on Azure or AWS.
• Use SQL, Python, and PySpark/Scala to transform and analyze large datasets.
• Collaborate with the team on Microsoft Power BI.
• Ensure best practices in development, testing, and deployment.
• Create technical documentation related to the project.

Skills

Data integration

Databricks

ETL/ELT

SQL

Python

Scala

PySpark

Communication in English

Education

Bachelor's degree in Computer Science or IT

Tools

Power BI

Databricks

Delta Lake

Azure

AWS

Job description

We are looking for an experienced Data Integration Engineer specializing in Databricks to join our team. You will design enterprise data products and pipelines that deliver value to users and achieve business goals. You will collaborate with various stakeholders and contribute to creating engaging and delightful solutions by leveraging your design skills and methods. Come and join us to promote the role of design and achieve positive design outcomes.

Responsibilities

• Develop, maintain and support ETL/ELT solutions
• Implement and manage Databricks pipeline/notebook on Microsoft Azure or AWS Cloud platforms
• Use SQL/ Python, and PySpark or Scala to transform and analyze large datasets
• Work collaboratively with the team on Microsoft Power BI
• Ensure the adherence to best practices in development, testing, and deployment
• Create technical documentation related to the project

Requirements

• Strong experience in Data Integration, proficient in Databricks, and ETL/ELT Solutions
• Strong SQL/Python skills for data transformation and analysis
• Experience with Delta Lake concepts (tables, partitions, schema evolution)
• Familiarity with cloud platforms (Azure, AWS)
• Have hands-on experience developing data pipelines on Databricks, integrating data from multiple sources, and working closely with analytics, data science, and business teams to deliver reliable, high-quality data products
• Knowledge of data ingestion techniques and data integration best practices
• Excellent verbal and written communication skills in English
• Bachelor’s degree, ideally in Computer Science, Information Technology, or a related discipline
• No visa sponsorship available

Nice to have

• Experience collaborating with international clients or counterparts, ideally including working with global teams

We offer

• By choosing EPAM, you're getting a job at one of the most loved workplaces according to Newsweek 2021 & 2022&2023.
• Employee ideas are the main driver of our business. We have a very supportive environment where your voice matters
• You will be challenged while working side-by-side with the best talent globally. We work with top-notch technologies, constantly seeking new industry trends and best practices
• We offer a transparent career path and an individual roadmap to engineer your future & accelerate your journey
• At EPAM, you can find vast opportunities for self-development: online courses and libraries, mentoring programs, partial grants of certification, and experience exchange with colleagues around the world. You will learn, contribute, and grow with us