Senior Databricks Data Engineer — ETL/Pipelines Lead

EPAM Systems Inc; Malaysia; Sur place; MYR 250 000 - 320 000; Plein temps

Résumé du poste

EPAM Systems Inc. seeks an experienced Senior Data Integration Engineer specializing in Databricks to lead the design of enterprise data products and platforms. You will own end-to-end data ingestion and transformation pipelines on Azure Databricks and mentor junior engineers.

Responsibilities include implementing Databricks on Azure or AWS, collaborating with analytics teams, and creating technical documentation. Strong English skills and a Bachelor's degree are required.

Qualifications

• At least 5 years of experience in Data Integration.
• Strong experience with Databricks and ETL/ELT.
• Proficient in SQL and Python; data modeling fundamentals.
• Experience with Delta Lake concepts.
• Familiar with Azure and AWS cloud platforms.
• Experience developing data pipelines on Databricks with multiple sources.
• Knowledge of data ingestion techniques and best practices.
• Orchestration experience (Databricks Workflows; ADF/Airflow a plus).

Responsabilités

• Develop, maintain and support ETL/ELT solutions.
• Implement and manage Databricks on Azure or AWS Cloud platforms.
• Own end-to-end delivery of data ingestion + transformation pipelines on Azure Databricks.
• Mentoring junior engineers.
• Work collaboratively with the team on Microsoft Power BI.
• Create technical documentation related to the project.

Connaissances

Databricks

ETL/ELT

SQL

Python

Data modeling

Delta Lake

Cloud platforms

Data pipelines

Orchestration

English communication

Formation

Bachelor's degree

Outils

Databricks Workflows

Azure Data Factory

Airflow

Description du poste

EPAM Systems Inc. seeks an experienced Senior Data Integration Engineer specializing in Databricks to lead the design of enterprise data products and platforms. You will own end-to-end data ingestion and transformation pipelines on Azure Databricks and mentor junior engineers.

Responsibilities include implementing Databricks on Azure or AWS, collaborating with analytics teams, and creating technical documentation. Strong English skills and a Bachelor's degree are required.