Senior Data Integration Engineer (Databricks)

EPAM Systems; Kuala Lumpur; On-site; MYR 180,000 - 280,000; Full time

Benefits offered by this job

Recognized workplace by Newsweek 2021-

Supportive environment where your voix

Work with top-notch technologies and a

Transparent career path and roadmap

Self-development: online courses, ment

Mentoring programs and certifications

Job summary

EPAM Systems is seeking an experienced Senior Data Integration Engineer specialized in Databricks to lead enterprise data product design and delivery. You will collaborate with stakeholders to create robust data solutions and data platforms that deliver business value.

You will develop and maintain ETL/ELT pipelines, manage Databricks on cloud platforms, and mentor junior engineers while working with analytics and business teams to deliver high-quality data products.

Qualifications

• ≥5 years in data integration with Databricks and ETL/ELT pipelines.
• Proficient in SQL and Python; strong data modeling (dimensional/Lakehouse).
• Experience with Delta Lake concepts and cloud platforms (Azure, AWS).
• Hands-on data pipelines on Databricks; collaborates with analytics and business teams.
• Familiar with orchestration (Databricks Workflows; Airflow/ADF a plus).
• Excellent written and verbal English communication; ability to mentor juniors.

Responsibilities

• Develop, maintain and support ETL/ELT solutions.
• Implement and manage Databricks on Azure or AWS.
• Use SQL, PySpark or Scala to transform and analyze large datasets.
• Own end-to-end delivery of data ingestion + transformation pipelines on Azure Databricks.
• Mentor junior engineers and collaborate with the team on Power BI.
• Create technical documentation; ensure best practices in dev/test/deploy.

Skills

Data integration

SQL

Python

Scala

ETL/ELT

Databricks

Power BI

English communication

Mentoring

Education

Bachelor's degree in CS/IT or related

Tools

Databricks

PySpark

Delta Lake

Airflow

Azure Data Factory (ADF)

Power BI

Job description

We are looking for an experienced Senior Data Integration Engineer specializing in Databricks to join our team. As a key member of the team, you will lead the design of enterprise digital products and platforms that deliver value to users and achieve business goals. You will collaborate with various stakeholders and contribute to creating engaging and delightful solutions by leveraging your design skills and methods. Come and join us to promote the role of design and achieve positive design outcomes.

Responsibilities

• Develop, maintain and support ETL/ELT solutions
• Implement and manage Databricks on Microsoft Azure or AWS Cloud platforms
• Use SQL, PySpark or Scala to transform and analyze large datasets
• Own end-to-end delivery of data ingestion + transformation pipelines on Azure Databricks
• Works independently with minimal supervision
• Mentoring junior engineers
• Work collaboratively with the team on Microsoft Power BI
• Ensure the adherence to best practices in development, testing and deployment
• Create technical documentation related to the project

Requirements

• At least 5 years of experience and strong experience in Data Integration, proficient in Databricks and ETL/ELT Solutions
• Strong SQL/Python, data modeling fundamentals (dimensional / Lakehouse modeling), incremental processing, data transformation and analysis
• Experience with Delta Lake concepts (tables, partitions, schema evolution)
• Familiarity with cloud platforms (Azure, AWS)
• Have hands-on experience developing data pipelines on Databricks, integrating data from multiple sources, and working closely with analytics, data science and business teams to deliver reliable, high-quality data products
• Knowledge of data ingestion techniques and data integration best practices
• Orchestration experience (Databricks Workflows; ADF/Airflow is a plus)
• Excellent verbal and written communication skills in English
• No visa sponsorship available

Nice to have

• Bachelor’s degree, ideally in Computer Science, Information Technology, or a related discipline

We offer

• By choosing EPAM, you're getting a job at one of the most loved workplaces according to Newsweek 2021 & 2022&2023.
• Employee ideas are the main driver of our business. We have a very supportive environment where your voice matters
• You will be challenged while working side-by-side with the best talent globally. We work with top-notch technologies, constantly seeking new industry trends and best practices
• We offer a transparent career path and an individual roadmap to engineer your future & accelerate your journey
• At EPAM, you can find vast opportunities for self-development: online courses and libraries, mentoring programs, partial grants of certification, and experience exchange with colleagues around the world. You will learn, contribute, and grow with us