Lead Data Engineer - Azure Databricks, Spark TB-scale ETL

PT Arbeit Infotech Indonesia; Taguig; On-site; PHP 1,000,000 - 1,600,000; Full time

Benefits offered by this job

Company Social Events

Bereavement Leave

Sick Leave

Wedding Leave

Work from Home

Job summary

Arbeit Group in Taguig, Philippines, is seeking an experienced Data Engineer to join a leading multinational company. You will build and maintain large-scale data pipelines using Azure Databricks and Spark, and design robust ETL workflows.

The role requires TB-scale data processing, strong Python/PySpark and SQL skills, and deep knowledge of Data Lakes, data quality, and governance. Collaboration with business users and cross-functional teams is essential.

Qualifications

• Minimum 5+ years of hands-on experience with Azure Databricks and Apache Spark.
• Strong Python, PySpark and SQL skills.
• Experience with TB-scale data processing and Data Lake architectures.
• Familiarity with Azure Data Analytics services and data governance.

Responsibilities

• Build, develop, and maintain large-scale data pipelines using Azure Databricks and Spark.
• Design and optimize ETL processes and data ingestion workflows.
• Develop high-performance Python/PySpark and SQL solutions for TB-scale datasets.
• Design and maintain modern Data Lake architectures.
• Ensure data quality, validation, integrity, and governance across data platforms.
• Utilize Azure data services for analytics, storage, and processing.
• Prepare technical documentation, requirements documents, and testing artifacts.
• Collaborate with business users, stakeholders, and technical teams to deliver data solutions.
• Support production environments and performance optimization initiatives.
• Manage source control and code versioning using Git.

Skills

Azure Databricks

Apache Spark

Python

PySpark

SQL

Data quality

ETL

Data governance

Stakeholder mgmt

Git

Education

Bachelor's degree

Tools

Git

Job description

Arbeit Group in Taguig, Philippines, is seeking an experienced Data Engineer to join a leading multinational company. You will build and maintain large-scale data pipelines using Azure Databricks and Spark, and design robust ETL workflows.

The role requires TB-scale data processing, strong Python/PySpark and SQL skills, and deep knowledge of Data Lakes, data quality, and governance. Collaboration with business users and cross-functional teams is essential.