Databricks Data Pipeline Architect (Azure/AWS)

EPAM Systems Inc; Malaysia; Vor Ort; MYR 120.000 - 240.000; Vollzeit

Zusammenfassung

EPAM Systems Inc. is seeking an experienced Senior Data Integration Engineer focused on Databricks to lead the design of enterprise data platforms. You will own end-to-end data ingestion and transformation pipelines on Azure Databricks, collaborate with analytics teams, and mentor junior engineers.

The ideal candidate has 5+ years in data integration, strong SQL/Python, Delta Lake knowledge, and hands-on Databricks experience across Azure or AWS. Excellent English communication is required.

Qualifikationen

• 5+ years of experience in Data Integration with strong Databricks/ETL/ELT skills.
• Proficient in SQL and Python, data modeling (dimensional/Lakehouse).
• Experience with Delta Lake concepts, schema evolution, partitions.
• Familiar with cloud platforms (Azure, AWS).
• Hands-on data pipelines on Databricks and integrating data from multiple sources.

Aufgaben

• Develop, maintain and support ETL/ELT solutions.
• Implement and manage Databricks on Azure or AWS.
• Transform and analyze large datasets with SQL, PySpark or Scala.
• Own end-to-end delivery of data ingestion and transformation pipelines on Databricks.
• Collaborate with analytics and business teams to deliver reliable data products.
• Mentor junior engineers and contribute to documentation.

Kenntnisse

SQL

Python

Data modeling

Data integration

Communication

Ausbildung

Bachelor's degree in Computer Science/Information Technology

Tools

Databricks

Delta Lake

Azure

AWS

Databricks Workflows

ADF / Airflow

Jobbeschreibung

EPAM Systems Inc. is seeking an experienced Senior Data Integration Engineer focused on Databricks to lead the design of enterprise data platforms. You will own end-to-end data ingestion and transformation pipelines on Azure Databricks, collaborate with analytics teams, and mentor junior engineers.

The ideal candidate has 5+ years in data integration, strong SQL/Python, Delta Lake knowledge, and hands-on Databricks experience across Azure or AWS. Excellent English communication is required.