Databricks Data Engineer

Unison; Singapore; On-site; SGD 70,000 - 90,000; Full time

Job summary

Unison is looking for a talented data engineer to build and optimize scalable data pipelines using Apache Spark on Databricks. The role requires collaboration with data scientists and maintaining distributed data jobs.

Responsibilities include implementing ETL/ELT processes and ensuring data quality and compliance. Proficiency in tools like Azure DevOps, GitHub Actions, and Terraform is essential. Join Unison in Singapore to contribute to impactful data products.

Qualifications

• Strong experience in building and optimizing data pipelines.
• Knowledge of ETL/ELT processes and data quality assurance.
• Familiarity with cloud migrations and infrastructure automation.

Responsibilities

• Build and optimize scalable data pipelines using Apache Spark on Databricks.
• Implement ETL/ELT processes with Databricks Notebooks and Delta Lake.
• Collaborate with data scientists and analysts to deliver data products.

Tools

Apache Spark

Databricks

Azure DevOps

GitHub Actions

Terraform

Job description

Key Deliverables:

• Build and optimize scalable data pipelines using Apache Spark on Databricks
• Implement ETL/ELT processes with Databricks Notebooks and Delta Lake
• Ensure data quality, integrity, and compliance across platforms
• Support CI/CD, cloud migrations, and infrastructure automation

Role Responsibilities:

• Collaborate with data scientists and analysts to deliver data products
• Maintain and optimize large-scale distributed data jobs
• Use Azure DevOps, GitHub Actions, and Terraform for workflow automation
• Work with structured and unstructured data using Lakehouse architecture