Data Engineer (Databricks)

Aryan-Solutions-Pte.-Ltd.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Aryan-Solutions-Pte.-Ltd. is seeking a senior Data Engineer with Databricks expertise to design catalogues, implement data pipelines and governance in a cloud data platform. You will optimize Spark workloads, manage CI/CD for Databricks artifacts and ensure observability across data flows.

The role requires at least 5 years in data engineering, 3 years with Databricks, and hands-on experience with Delta Lake, Unity Catalog, and PySpark to deliver scalable data solutions.

Qualifications

• Hands-on experience with Unity Catalog for catalogue design, access control and lineage.
• Extensive Delta Lake/Delta Live Tables for pipeline orchestration and data formats.
• Deep knowledge of Databricks SQL and cluster/workspace administration.
• Experience with Databricks Asset Bundles and Databricks Repos for CI/CD.
• Strong PySpark or Spark SQL for large-scale transformations.
• Observability via system tables, logs, billing/usage and query history.
• Minimum 5 years in Data Eng / Data Platform Eng roles; minimum 3 years with Databricks for pipelines and governance.

Skills

Unity Catalog

Data lineage

Delta Live Tables

Databricks SQL

CI/CD Databricks

Asset Bundles

Databricks Repos

PySpark

Observability

SQL

Python

Data modeling

Dimensional design

Star/Snowflake schemas

CI/CD pipelines

Terraform

Cloud (AWS/Azure/GCP)

Tools

Delta Lake

Lakeflow

Terraform

Azure DevOps

GitHub Actions

GitLab CI

Job description

Required Technical Skills — Databricks

• Unity Catalog — catalogue/schema design, access control, and data lineage
• Lakeflow / Delta Live Tables for pipeline orchestration; Delta Lake table format
• Databricks SQL and cluster/workspace administration (compute policies, pools, cost management)
• Databricks Asset Bundles (DABs) and Databricks Repos for CI/CD
• PySpark / Spark SQL for large-scale data transformation
• Working knowledge of Databricks system tables (audit logs, billing/usage, query history) for observability
• Minimum 5 years of hands-on experience in Data Engineering, Data Platform Engineering, or related disciplines.
• Minimum 3 years of hands-on experience with Databricks involving data pipeline development, platform administration, governance, and optimization.

Required Technical Skills — Common

• Strong SQL and Python (PySpark or general-purpose) for data engineering
• Data modeling — dimensional design, star/snowflake schemas, semantic layers
• CI/CD pipelines (e.g., Azure DevOps, GitHub Actions, GitLab CI) for data engineering workflows
• Infrastructure-as-code (Terraform preferred) for provisioning cloud data platform resources
• Hands-on experience on at least one hyperscaler — AWS, Azure, or Google Cloud

Understanding of data security and compliance frameworks applicable to government/public-sector environments