Data Engineer: PySpark, Databricks & Delta Lake

APAR TECHNOLOGIES PTE. LTD.; Singapore; On-site; SGD 90,000 - 150,000; Full time

Job summary

APAR TECHNOLOGIES PTE. LTD. in Singapore is seeking a data engineer to design and maintain scalable data pipelines. You will use PySpark with Databricks to process data, implement Delta Lake with SQL transformations, and optimize performance for reliability.

The role involves integrating data from APIs, databases and files, enforcing data quality checks, and collaborating with analysts and business teams to meet data requirements.

Qualifications

• 3–5 years of relevant data engineering experience.
• Strong knowledge of Python, PySpark and SQL.
• Hands-on experience with Databricks and Delta Lake.
• Experience with Azure, AWS or GCP.
• Knowledge of data modelling, ETL/ELT, Git and CI/CD.

Responsibilities

• Develop and maintain data pipelines using PySpark and Databricks.
• Process and transform data using Delta Lake and SQL.
• Optimize pipelines for performance and reliability.
• Integrate data from APIs, databases and files.
• Implement data quality checks and monitor pipelines.
• Collaborate with analysts and business teams to meet data requirements.

Skills

Python

PySpark

SQL

Data engineering

ETL/ELT

Tools

Databricks

Delta Lake

Azure

AWS

GCP

Git

CI/CD

Unity Catalog

Airflow

Azure Data Factory

Kafka

Terraform

dbt

Job description

APAR TECHNOLOGIES PTE. LTD. in Singapore is seeking a data engineer to design and maintain scalable data pipelines. You will use PySpark with Databricks to process data, implement Delta Lake with SQL transformations, and optimize performance for reliability.

The role involves integrating data from APIs, databases and files, enforcing data quality checks, and collaborating with analysts and business teams to meet data requirements.