Data Engineer: Spark, Databricks & Azure

ConnectOS; Mandaluyong; On-site; PHP 900,000 - 1,400,000; Full time

Benefits offered by this job

Medical coverage

Paid vacation leave

Competitive salary package

Financial assistance program

13th month pay

Office perks

Job summary

ConnectOS is seeking a data engineer to design, develop, and support scalable data pipelines using Apache Spark and Databricks. You will manage data in Parquet and Delta Tables, participate in governance with Unity Catalog, and collaborate on data models with senior engineers.

You should be proficient in Python and familiar with Azure data services (ADF, ADLS, Synapse). The role emphasizes cloud fundamentals, security, and scalable architectures within a Monday–Friday schedule in Manila.

Qualifications

• Proven experience in data engineering, including the design, development, and support of scalable data pipelines.
• Strong understanding of Parquet, Delta Lake and Delta Tables formats.
• Hands-on experience with Apache Spark and DataFrame APIs for data processing and transformation.
• Familiarity with Databricks notebook development, workflows, and scheduled jobs.
• Knowledge of data governance and Unity Catalog is advantageous.
• Proficient in Python for clean, maintainable code and collaboration on codebases.
• Experience with Microsoft Azure data services (ADF, ADLS, Synapse, etc.).
• Solid cloud computing fundamentals including scalability, cost optimisation, and security.

Responsibilities

• Build and maintain data pipelines using Spark and Databricks.
• Work with Parquet and Delta Table formats for structured data.
• Assist with Unity Catalog governance and access control.
• Collaborate with architects to implement data models and transformation logic.
• Write maintainable Python code and participate in testing and reviews.
• Investigate data quality and performance issues with the team.

Skills

Data engineering

Apache Spark

Python

Databricks

Unity Catalog

Cloud fundamentals

Tools

Parquet

Delta Lake

Delta Tables

Databricks notebooks

Azure Data Factory

Azure Data Lake Storage

Synapse Analytics

Git

Azure DevOps

Job description

ConnectOS is seeking a data engineer to design, develop, and support scalable data pipelines using Apache Spark and Databricks. You will manage data in Parquet and Delta Tables, participate in governance with Unity Catalog, and collaborate on data models with senior engineers.

You should be proficient in Python and familiar with Azure data services (ADF, ADLS, Synapse). The role emphasizes cloud fundamentals, security, and scalable architectures within a Monday–Friday schedule in Manila.